Image interpolation, as a method of obtaining a high-resolution image from the corresponding low-resolution image, is a classical problem in image processing. In this paper, we propose a novel energy-driven interpolation algorithm employing Gaussian process regression. In our algorithm, each interpolated pixel is predicted by a combination of two information sources: first is a statistical model adopted to mine underlying information, and second is an energy computation technique used to acquire information on pixel properties. We further demonstrate that our algorithm can not only achieve image interpolation, but also reduce noise in the original image. Our experiments show that the proposed algorithm can achieve encouraging performance in terms of image visualization and quantitative measures.
Image interpolation is a very important aspect of image processing and involves the use of a known pixel set to produce an unknown pixel set, resulting in an image of higher resolution [
Overview of our approach for image interpolation.
Our contribution is twofold. Firstly, we propose a framework for both magnification and deblurring in order to fulfill the interpolation task for low-resolution images with low noise. Secondly, we demonstrate an energy-driven approach based on the properties of adjacent pixels within this framework. In addition, we define the processing unit and its properties for better implementation of the EGPR algorithm.
The rest of the paper is structured as follows. Section
In recent years, GPR has become a hot issue in the field of machine learning and has attracted great academic interest [
There are a variety of covariance functions, of which one of the most commonly used is the squared exponential (SE) covariance function
We can make use of Gaussian identities to obtain (
In this paper, we combine the energy-driven approach with GPR to accomplish the task of image interpolation. The proposed algorithm models low-resolution image data as a function of a probability distribution that satisfies a local static Gaussian process. This algorithm framework is shown in Figure
Architecture of the proposed algorithm.
The following definitions are used in the EGPR algorithm.
A given image
Given a total of
To facilitate the operation of the PU, it is necessary to introduce some properties in advance.
Given a number
Given
Denoising is the first step in the EGPR algorithm, and we use the following formula (
Before applying GPR, we can obtain the particular relationship between the input and output vectors of PU according to Properties
Training plays an important role in the EGPR algorithm, and we adopt a different approach from that used in [
Given two processing units
When the search step count reaches the predefined number, or if the PU structure similarity falls below a certain value, the first stage is complete. In the second stage, we apply a Gaussian process prior probability and establish the GPR model with Gaussian noise
When aiming to achieve high-quality images, the conjugate gradients method is chosen to obtain the model hyperparameters, including mean, variance, and log marginal likelihood. Notice that different iteration numbers in the conjugate gradients method may lead to different prediction accuracies. Figure
Images obtained after adaptation with different numbers of iterations. In (a), many black points are observed, each indicating a zero prediction for the pixels. In (b), the black points have been eliminated.
Iteration = 50
Iteration = 100
Inspired by the ICBI algorithm, we firstly compute the initial pixel value
However, the pixel value obtained is only a roughly estimated value and needs further refinement. Following [
Suppose that the low-resolution image
Similarly, we partition
During the prediction of high-resolution image pixels, two rules should be obeyed. Firstly, the PU divided by the initial high-resolution image should correspond to that divided by the low-resolution image. Secondly, the gradient algorithm should satisfy the common positive definite matrix. If not, it will lead to a zero prediction, and the prediction value will need modifying. The modification method can be utilized to maintain the original interpolated pixel value. Finally, we combine all the processing units together in a smooth manner to obtain the high-resolution images without noise.
In this section, we compare the experimental results obtained using the proposed algorithm with those obtained using the bilinear algorithm, GPR algorithm [
Figure
Comparison of images obtained using four methods, with scale = 1. Parts (a)–(d) show image 1. Parts (e)–(h) show image 2.
Bilinear
GPR
ICBI
EGPR
Bilinear
GPR
ICBI
EGPR
Similarly, Figures
Comparison of images obtained using four methods, with scale = 2. Parts (a)–(d) show image 3. Parts (e)–(h) show image 4.
Bilinear
GPR
ICBI
EGPR
Bilinear
GPR
ICBI
EGPR
Comparison of images obtained using four methods, with scale = 3. Parts (a)–(d) show image 5. Parts (e)–(h) show image 6.
Bilinear
GPR
ICBI
EGPR
Bilinear
GPR
ICBI
EGPR
To further validate our algorithm, we also provide objective measurements. Peak signal-to-noise ratio (PSNR) and root mean square (RMS) error are traditional quantitative measures of accuracy, and by comparing their values for the above images, we can conclude that the proposed EGPR algorithm yields interpolated pixel values that are much closer to their original high-quality values than those obtained with the bilinear algorithm, GPR algorithm, and ICBI algorithm. Tables
Comparison of PSNR for the four interpolation methods when applied to test images.
Image | Scale | Bilinear | GPR | ICBI | EGPR |
---|---|---|---|---|---|
Image 1 | 1 | 32.9940 | 33.2792 | 33.3456 | 33.3986 |
Image 2 | 1 | 30.6314 | 30.7861 | 31.3684 | 31.4594 |
Image 3 | 2 | 29.5738 | 29.4194 | 29.7173 | 29.7213 |
Image 4 | 2 | 27.7717 | 27.4767 | 27.8485 | 27.8625 |
Image 5 | 3 | 23.4038 | 24.4366 | 24.7153 | 24.7171 |
Image 6 | 3 | 24.3122 | 25.1477 | 25.6880 | 25.6909 |
Comparison of RMS for the four interpolation methods when applied to test images.
Image | Scale | Bilinear | GPR | ICBI | EGPR |
---|---|---|---|---|---|
Image 1 | 1 | 16.4437 | 15.8419 | 15.7032 | 15.6004 |
Image 2 | 1 | 21.1046 | 20.3410 | 19.1890 | 18.9614 |
Image 3 | 2 | 24.9225 | 24.8861 | 24.4329 | 24.4191 |
Image 4 | 2 | 31.0516 | 32.1427 | 30.6882 | 30.6118 |
Image 5 | 3 | 50.8161 | 44.8762 | 43.3017 | 43.2633 |
Image 6 | 3 | 45.9833 | 41.5720 | 39.0571 | 39.0412 |
MSSIM [
Comparison of MSSIM for the four interpolation methods when applied to test images.
Image | Scale | Bilinear | GPR | ICBI | EGPR |
---|---|---|---|---|---|
Image 1 | 1 | 0.936 | 0.937 | 0.938 | 0.940 |
Image 2 | 1 | 0.946 | 0.947 | 0.953 | 0.955 |
Image 3 | 2 | 0.905 | 0.906 | 0.909 | 0.910 |
Image 4 | 2 | 0.812 | 0.808 | 0.815 | 0.816 |
Image 5 | 3 | 0.818 | 0.837 | 0.850 | 0.851 |
Image 6 | 3 | 0.857 | 0.865 | 0.878 | 0.879 |
In addition, Figure
Quantitative quality assessment results for the four interpolation methods.
In this paper, we have presented a novel EGPR method for image interpolation. The main feature of this new algorithm is its ability to obtain relatively high prediction accuracy of the unknown pixels by fully utilizing underlying image patch information. The implementation process involves two steps: training and prediction. The former creates a GPR model using only single-image data as the training set, and the latter combines energy computation with the acquired model to produce a high-resolution image. Experiments have shown that our algorithm can yield encouraging performance not only in terms of image visualization but also in terms of PSNR, RMS, and MSSIM quality measures. However, better image interpolation comes at the expense of greater algorithm complexity. Methods of improving the algorithm efficiency need further investigation. In future, we can improve this algorithm to address the problem of the interpolation of image sequences. Images in the same sequence are also subject to the recurrence phenomenon, whereby images contain spatial-temporal correlation [
This work was supported by the National Basic Research Program of China (973 Program) 2012CB821200 (2012CB821206), the National Natural Science Foundation of China (no. 91024001, no. 61070142), and the Beijing Natural Science Foundation (no. 4111002).