^{1}

^{1}

^{2}

^{1}

^{1}

^{2}

In view of the drawback of most image inpainting algorithms by which texture was not prominent, an adaptive inpainting algorithm based on continued fractions was proposed in this paper. In order to restore every damaged point, the information of known pixel points around the damaged point was used to interpolate the intensity of the damaged point. The proposed method included two steps; firstly, Thiele’s rational interpolation combined with the mask image was used to interpolate adaptively the intensities of damaged points to get an initial repaired image, and then Newton-Thiele’s rational interpolation was used to refine the initial repaired image to get a final result. In order to show the superiority of the proposed algorithm, plenty of experiments were tested on damaged images. Subjective evaluation and objective evaluation were used to evaluate the quality of repaired images, and the objective evaluation was comparison of Peak Signal to Noise Ratios (PSNRs). The experimental results showed that the proposed algorithm had better visual effect and higher Peak Signal to Noise Ratio compared with the state-of-the-art methods.

Image inpainting is an important branch of image processing which studies how to restore the damaged part based on human visual mechanism. Currently, there are many image inpainting methods, including methods based on partial differential equation (PDE) [

PDE is introduced to propagate local structures from the exterior to the interior of the damaged regions. Zhou and Gao [

The texture synthesis scheme is another effective method for image inpainting [

With the advent of sparse representation, sparse priors have also been used to solve the inpainting problem. Hu and Xiong [

Huo et al. [

These methods above have good repaired effect, however, by which texture details cannot be well processed. Considering that the reconstruction images have better visual effect and prominent texture by the continued fractions [

The main contributions of this paper are as follows: the adaptive inpainting scheme based on Thiele’s rational interpolation is proposed; the novel inpainting model by Thiele’s rational interpolation combined with Newton-Thiele’s rational interpolation is proposed.

In this section, we mainly summarize the proposed method. Our method consists of two phases, namely, adaptive inpainting phase and refining inpainting phase. In the adaptive inpainting phase, we adopt Thiele’s rational interpolation function and the corresponding mask image to interpolate the intensities of damaged pixel points. According to the mask image, we can judge the overall direction of scratches. If the overall direction of scratches is horizontal, then the damaged pixel points will be processed in columns; otherwise, they are processed in rows. For example, the overall direction of scratches is vertical in Figure

Flowchart of the proposed method. For a damaged image, we use Thiele’s rational interpolation function and the mask image to get an initial repaired image. And then, by using Newton-Thiele’s rational interpolation function, interpolation window, and the mask image, the initial repaired image is refined to get a final result.

In the refining inpainting phase, we adopt Newton-Thiele’s rational interpolation function to update the intensity of every damaged point. After the first phase, the damaged points have been repaired, and, in the second phase, we will refine the previous result in order to be closer to the original image. We use the mask image again to find the position of every damaged point corresponding to the initial repaired image. Different from previous continued fractions inpainting method [

The continued fractions are an ancient branch of mathematics, and the theory appeared for the first time in 1948 [

Suppose a set of real or complex points

It is not difficult to show that

From the above section, we find that when the denominator is zero and the inverse differences

For image inpainting by Thiele’s rational interpolation, the key is the selection of interpolation sampling points. Considering the computational complexity of continued fractions, we adaptively select 4 sampling points for interpolation. These sampling points are all known pixel points that are close enough to the damaged point being interpolated. The selection order of sampling points is as shown in Figure

Selection of sampling points.

The proposed inpainting algorithm by Thiele’s rational interpolation can work only when there is consistent scratching direction in damaged domain. As shown in Figure

A scratching simulated diagram.

Now we describe the adaptive inpainting algorithm in detail. From Figure

Inpainting schematic diagram.

Newton-Thiele’s rational interpolation is formed jointly by Newton’s polynomial in

Let

Then it is not difficult to show that

We find that if

Different from Thiele’s rational interpolation, a

By using Thiele’s interpolation inpainting algorithm, we get an initial repaired image, and next we need to refine the intensity of every damaged pixel point. Because every damaged point has been repaired initially, in the refinement process, except for the damaged point being interpolated, the other damaged points are treated as known pixel points whose intensities are those of the initial repaired image. The mask image is used again to determine the position of every damaged pixel point, and the interpolation window in Figure

In order to get a better repaired image, we use Thiele’s rational interpolation to get an initial repaired image, and then Newton-Thiele’s rational interpolation is used to refine the initial repaired image to get a final result. The whole inpainting algorithm includes two steps: the adaptive inpainting and the refining inpainting. In the adaptive inpainting phase, Thiele’s rational interpolation function is used to interpolate intensity of every damaged point, and the detailed process is summarized in Algorithm

In the refining inpainting phase, Newton-Thiele’s rational interpolation function is used to refine the intensity of every damaged point, and the detailed process is summarized in Algorithm

Peak Signal to Noise Ratio (PSNR) is often used as a measure of signal reconstruction quality, which can be defined as follows:

In this section, we demonstrate the effectiveness and superiority of the proposed method through plenty of experiments. In our experiments, the original images are selected from standard datasets (set 5, set 14, and B100) and the original images with scratching are used as damaged images. The standard datasets are used for image processing, which can be downloaded from the Internet (

From Figures

The comparison of PSNRs.

PSNR | BSCB | CT | EB | TV | Thiele | Ours |
---|---|---|---|---|---|---|

Figure | 33.299020 | 33.196864 | 30.390751 | 29.572404 | 33.087954 | |

Figure | 34.034243 | 33.838789 | 31.795258 | 28.643684 | 33.902315 | |

Figure | 20.082217 | 20.419491 | 17.850016 | 20.41844 | 20.748204 | |

Figure | 32.907370 | 35.794873 | 33.724928 | 28.595606 | 35.921904 | |

Figure | 25.846801 | 27.711887 | 24.733383 | 26.424346 | 26.993011 | |

Figure | 23.084520 | 24.303285 | 18.296882 | 16.128918 | 23.808367 | |

Repaired hat images by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

Repaired results by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

Repaired cartoon images by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

Repaired girl images by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

Repaired parrot images by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

Repaired raccoon images by different inpainting methods. (a) The damaged image; (b) the mask image; (c) by BSCB; (d) by CT; (e) by EB; (f) by TV; (g) by Thiele; (h) by our method; (i) the original image.

In order to illustrate the property of prominent texture details by our method, we select repaired image patches from Figure

The texture patches display of repaired raccoon images by different inpainting methods. (a) By BSCB; (b) by CT; (c) by EB; (d) by TV; (e) by Thiele; (f) by our method; (g) the original image.

The intensity distribution of one column of repaired image patches. (a) By BSCB; (b) by CT; (c) by EB; (d) by TV; (e) by Thiele’s interpolation; (f) by our method.

Through visual comparison of Figures

A novel image inpainting method by using nonlinear rational interpolation was presented. Our approach was based on the observation that the texture details of repaired images by most image inpainting algorithms were not prominent. Inspired by the applications of continued fractions in image processing [

Fully documented templates are available in the elsarticle package on CTAN.

The authors declare that they have no conflicts of interest.

This work is supported by the National Natural Science Foundation of China (Grants nos. 61502141, 61070227, 61472466, and 11601115), the Anhui Provincial Natural Science Foundation (Grant no. 1508085QF128), and the Fundamental Research Funds for the Central Universities (Grants nos. JZ2015HGXJ0175 and JZ2016HGBZ1005).