Accurate Quaternion Polar Harmonic Transform for Color Image Analysis

,


Introduction
Rotation-invariant moments (RIMs) are extensively used in image representation and pattern recognition [1][2][3] because of their outstanding description capability and invariance property.Moreover, RIMs not only provide hand craft features for image representation but also can reconstruct original image from these hand craft features, making them suitable for image watermarking [4][5][6].ere exist two kinds of RIMs: orthogonal and nonorthogonal.Since the orthogonal RIMs (ORIMs) possess minimum information redundancy and hence better information compactness, they are effective in performance.
ere are several low order moments and transforms can sufficiently extract the essential features of images.e most popular among these ORIMs [7,8] are Zernike moments (ZMs) and pseudo ZMs (PZMs).Yap et al. [9] recently introduced a few ORITs, collectively known as PHTs, including the polar sine transforms (PSTs), polar cosine transforms (PCTs), and polar complex exponential transforms (PCETs).ORIMs are different from ORITs in the radial parts of kernel functions, which are polynomials and sinusoidal functions in ORIMs and ORITs, respectively.Compared with ORIMs, PHTs are more efficient in computation [10].Besides, the high order transforms of PHTs are numerically stable, whereas those of ORIMs are numerically unstable.
erefore, PHTs are preferred to ORIMs and have recently been utilized in numerous image processing applications.
Especially for small size images, the current computational framework of PHTs is disadvantaged by geometric errors and numerical integration errors, which can be seen from image reconstruction errors, which are visible near the circular disk center, and even low orders of transform are numerically unstable.Mapping a square image into a unit circular disk can cause geometric errors, and approximating the integration with zeroth-order summation will lead to numerical integration errors.Accurate PHTs are extremely essential in tremendous image processing applications, primarily in template matching and optical character recognition problems using small size images.In addition to the problem of computation accuracy, another problem is that ORIMs and ORITs mainly focus on grayscale images, yet with the continuous improvement of computer performance, color images are drawing more and more attention from researchers in the related fields, because color images can provide much more abundant information compared with grayscale images.Most of the current research on color image moments depends on the intensity or a single channel within the color space of color images, where the information and relationship between color components in a specific color space are discarded.Several traditional orthogonal and nonorthogonal moments have been developed into quaternion moments, e.g., quaternion Zernike moments (QZMs) [2], quaternion radial harmonic Fourier moments (QRHFM) [11], quaternion polar harmonic Fourier moments (QPHFMs) [12], and quaternion polar harmonic transforms (QPHTs) [13,14], Although various quaternion image orthogonal moments have been proposed by researchers, most of them give unsatisfied image reconstruction performance.In the present work, accurate quaternion orthogonal transforms, i.e., accurate QPHTs (AQPHTs), are proposed.In summary, the innovations of the proposed AQPHTs include the following: firstly, by using the algebra of quaternions, we extend the PHTs to color images.Secondly, a computational framework, which reduces geometric errors and numerical integration errors, is developed to construct an approach to accurately calculate PHTs.Experiments are conducted to comparatively study the performance in image reconstruction and image retrieval of AQPHT and other ORIMs, and the results present that AQPHT has the best image reconstruction performance and superb behavior in invariant image representation with and without noises.
e paper is organized as follows.Section 2 introduces the quaternion algebra and PHTs.Section 3 describes the computational framework of our AQPHTs.In Section 4, the effectiveness of the proposed AQPHTs is evaluated.Section 5 concludes this study.

Preliminaries
2.1.Quaternion Algebra.A quaternion can be regarded as a generalized complex number.e formula of a quaternion is as follows: where q denotes the quaternion, a and bi + cj + dk denote the real and the imaginary part; a, b, c, and d are real numbers, and i, j, and k are complex operators that satisfy these conditions: e conjugate of μ is μ � a − bi − cj − dk, and the magnitude equals |μ| � . A quaternion μ � s(μ) + v(μ) can be considered as the combination of a scalar part s(μ) � a and a vector part v(μ) � bi + cj + dk.

Polar Harmonic Transform.
e two-dimensional polar harmonic transforms M n,l of an n-order image function f(r, θ), with repetition l(|l| ≥ 0) defined by where Ω n denotes a normalization factor, and H * n,l (r, θ) denotes the complex conjugate of the basis function H n,l (r, θ).For PCET, Ω n � 1/π and H n,l (r, θ) are given by . While in PCT and PST, e basis functions of PST and PCT are defined, respectively, as e total number of PCET for n � n max and l � l max is (1 + 2n max )(1 + 2l max ) where n max � l max .However, PST and PCT have a smaller number of features, which are For an N × N image, there is no analytical solution to the integration in equation ( 3), therefore its zeroth-order approximation is commonly used and given by where 2 Scientific Programming We perform inverse transforms to reconstruct the image function.It is expressed by where n max , n min , l max , and l min are the maximum and minimum orders and repetitions of PHTs, respectively, as specified in equation (3).

Accurate QPHTs
3.1.QPHTs.In traditional processing methods, color images are usually divided into 3 components and each component is separately handled by a follow-up process without taking into account the association within components.But, quaternion-based theory treats a color image as an integral vector, by which the relationships between each component are reflected.Let f R (r, θ), f G (r, θ), and f B (r, θ) denote the color components of red, green, and blue, respectively, then we encode a color pixel f(r, θ). e pixel can be encoded as a pure quaternion: erefore, PHTs can be defined in the quaternion field.e multidimension feature of color images can be addressed by the transforms wherein each color pixel is treated as a whole.Due to the noncommutative multiplication of quaternions, each quaternion transform has two different forms.e left-side quaternion PCET is defined as follows: where μ denotes a pure unit quaternion and f(r, θ) denotes the quaternion representation of a color pixel.
, i.e., the gray line in the RGB space is chosen in this work.By reversing the orders of images and transform kernels in equation (11), we can obtain the rightside QPCET: e reconstruction of a color image by using the QPCET coefficients of the left and right sides can be written as equations ( 13) and ( 14), respectively: Analogous to QPCET, the left and right-side QPCT can be defined as equations ( 15) and ( 16), respectively, where all factors are defined as in equation ( 5).Replace the radial components in Equations ( 15) and ( 16) with f B (r, θ), and we can obtain the QPST.e construction of color images can also be achieved by using QPCT and QPST coefficients.
We note that the QPHT in this paper refers to the leftside QPCET M L n,l as defined in equation ( 11), unless otherwise specified.

Computation of Accurate QPHT.
is subsection provides a computational framework for calculating QPHTs, which can reduce geometric errors and numerical integration errors.Using our method, only the part inside the unit disk of the pixel is considered.For numerical integration, we rewrite equation (11) as We assume the image function f B (r, θ) is constant in one grid.
e calculation accuracy of QPHTs can be increased through computing the numerical integration of basis function h nl,jk for each grid, that is, where Since equation ( 18) is evaluated by using a g × g point Gaussian numerical integration method, it can be simplified into e values of ω and t for a given g can be obtained by using standard procedures [15].For a quick reference, we provide these values in Table 1 for g � 1 through 8. e constraint in equation ( 21) is an improvement over the constraint x 2 j + y 2 k ≤ 1 in the zeroth-order approximation.If the center of a grid falls outside the unit circle, this grid will be completely ignored in computation in zerothorder approximation.But in the new constraint, a grid will be considered in the calculation if its sampling point falls within the unit disk even if its centers fall outside the unit disk.An improvement of the approximation of the unit disk has been observed with this new constraint.is is a great contribution of the numerical integration method for reducing geometric errors and numerical integration errors simultaneously by ensuring that none of the sampling points fall outside the unit disk.We find that the performance of image reconstruction improves as we increase the number of g, but the gain starts to saturate at g � 6.
erefore, the accurate computation experiment of QPHTs uses 5 × 5 sampling points (i.e., g � 5), which we take as a tradeoff accuracy and speed for color images.

Geometric Invariance of AQPHT.
Here, we will derive and analyze the rotation and scaling invariant property of AQPHTs.

Rotation Invariance. Let 􏽢
f(r, θ) � f(r, θ + α) denote the image f(r, θ) rotated by the angle α.Accordingly, the left-side AQPHT of  f(r, θ) is where M n,l and  M n,l are the AQPHTs of f(r, θ) and  f(r, θ), respectively.
Accordingly to equation ( 21), we know that a rotation of the color image by an angle α induces a phase shift e μlθ of the M L n,l (f).Taking the norm on both sides of equation ( 17 erefore, the rotation invariant can be achieved by taking the norm of APHTs.In other words, the AQPHTs modulus coefficients |M L n,l (f)| are invariant with respect to image rotation.

Scaling Invariance.
eoretically, AQPHTs are not invariant to image scaling, but scaling invariance can be obtained by normalizing the image into a unit circle.If an image f(r, θ) with N × N pixels is mapped to a unit circle (x j , y k ) ∈ [−1, 1] × [−1, 1] with x j � 2j − N + 1/N, y k � N − 1 − 2k/N, (j, k � 0, 1, . . ., N − 1) and a unit circle is made to cover the same contents of the image, the AQPHTs are variant to image scaling.

Experiments and Analysis
is section is intended to validate the effectiveness of AQPHT invariants for color images.e experiments are performed using MATLAB version 8.6 on a 2.9 GHz processor, 8 GB RAM computer, and Microsoft Windows 10 Ultimate operating system.

Experiment on Scaling
Invariance.Numerous great classification models, e.g., VGG [16] and ResNet [17], have been developed on the basis of the powerful deep learning framework eir recognition accuracy is amazing and even higher than that of human beings.However, the performance is achieved only for large size images having rich object structure and high quality appearance.e resolution of images with small size is low, which limits the learning of discriminative representations, thus leading to identification failure [18].In this experiment, the impact of down-scaling operation on image representations can be evaluated on Oxford5K dataset [19] by our AQPHT and the widespread deep model of VGG16.For convenience, only the neural activation in the 36-th layers of the VGG16 model is studied as an example of deep models in this work.e ability of is subsection compares the performance of image reconstruction between AQPHT and QZM [2], QPHT [14], QPHFM [12], and QRHFM [11].Following the standard [11,12], we set the relevant parameters in our experiments.e mean square reconstruction error (MSRE) is used to measure the reconstruction performance [11,12]; lower MSREs equate to better reconstruction performance.Suppose f(x, y) and  f(x, y) indicate the original image and reconstructed image, respectively; the MSRE is defined as In Figure 2, we plot the curves of average MSREs versus the data amount of coefficients.e coefficients in image reconstruction can be represented by real values, and the total number of required real values is defined as data amount.For instance, four real values are needed to represent one AQPHT coefficient, while representing the coefficients of three components of APHT-RGB require six real values [9].It can be seen from Figure 2 that our AQPHT achieves higher MSREs than APHT-RGB.By dealing with various color channels in a holistic way, AQPHTs can obtain the essence of color images from the inter-and intrachannel directions, demonstrating better compactness.
Next, we use two additional metrics, i.e., PSNR and SSIM, to evaluate the reconstruction performance.e PSNR between the reconstructed image  Y and original image Y of size is calculated as where MAX I is the maximum possible pixel intensity value.e SSIM is designed to better match the human perception compared to PSNR; the SSIM is defined as where μ and σ are respectively the average and variance of the pixel values, σ  YY is the covariance of  Y and Y; c 1 and c 2 are constants.Following previous methods [20,21], we set c 1 � 0.01 and c 2 � 0.03 in this paper.Higher values for PSNR and SSIM indicate better performance.
e "Lena" of size of 128 × 128 is reconstructed using our AQPHT and the compared methods with a maximum order ranging from 4 to 28.As shown in Figure 3, we can find that the reconstructed images of our AQPHT are far better than other moments.When the number of moments exceeds a certain value, the reconstruction performance of QZM [2] and QPHT [14] even degrades, whereas the reconstruction performance of our proposed AQPHT keeps getting better.

Experiment on Image Retrieval.
Here, a series of extensive experiments are conducted to compare our method with other leading edge ones, namely, traditional hand craft-based methods [22][23][24][25] and CNN-based methods [26][27][28][29][30]. To evaluate the performance, we use the average precision (AP) measure computed as the area under the precision-recall curve for a query.We compute an average precision score for each of the 5 queries for a landmark, averaging these to obtain a mean average precision (mAP) score.e average of these mAP scores is used as a single number to evaluate the overall performance.For a fair comparison, postprocessing methods, e.g., query expansion are excluded in the work, and only mAP in representation with relevant feature dimensions is reported.e results of retrieval accuracy (mAP) [31] of the UK-bench [32], Holidays [33], and Oxford5k [19] are presented in Table 2, in which the bold indicates the best results.From Table 2, the performance of our method is much better than all hand craft-based methods but marginally worse than some CNN-based methods [28][29][30].However, the length of a feature vector largely determines its retrieval efficiency.We note that our method uses feature vectors with significantly low dimensions comparing with CNN-based methods.In other words, the retrieval efficiency of our method is increased under the same experimental conditions in spite of the slight decrease of its accuracy, which means our method is a more effective compromising method for color image retrieval.Besides, the proposed AQPHT performs better than the QPHT [31]    large margin, which demonstrates the image representation ability of AQPHT which is stronger than that of QPHT.

Conclusion
is paper presents a novel computational framework of quaternion-based polar harmonic transform, namely, accuracy quaternion polar harmonic transforms (AQPHTs).Firstly, to holistically deal with color images, AQPHTs are introduced based on the algebra of quaternions.Secondly, geometric errors and numerical errors are reduced by using the Gaussian numerical integration.Many comparative experiments are conducted to analyze the performance of AQPHTs and other ORIMs.Experimental results verify the superb performance of AQPHTs in image reconstruction and invariant image representation.For future work, AQPHTs would be tested in other color image processing domains, e.g., watermarking, segmentation, and retrieval.Besides, more accurate algorithms will be put forward and the computational method of quaternion moments will be improved.

Table 1 :
Weights and locations of sampling points for g × g point integration.

Table 2 :
Comparison of retrieval performance in terms of mean average precision (mAP).