This paper introduces a new framework for image coding that uses image inpainting method. In the proposed algorithm, the input image is subjected to image analysis to remove some of the portions purposefully. At the same time, edges are extracted from the input image and they are passed to the decoder in the compressed manner. The edges which are transmitted to decoder act as assistant information and they help inpainting process fill the missing regions at the decoder. Textural synthesis and a new shearlet inpainting scheme based on the theory of
Image inpainting [
Chan et al. [
In this paper, a new discrete multiscale representation [
This paper is arranged as follows. First, in Section
A standard image model [
The standard wavelet transform of
Damages in the wavelet domain cause loss of wavelet coefficients of
For inpainting when we used the traditional wavelets [
The discrete shearlet transform of
When this model is solved, the use of TV norm can retain sharp edges while reducing noise and other oscillations. But the corresponding Euler-Lagrange equation is not trivial to compute since it is highly nonlinear and ill-posed in strong sense. Furthermore, this model has the drawback that is called stair case effect. To overcome these deficiencies, a
The Euler-Lagrange equation of the above inpainting model is
The gradient descent flow of (
The above equation is solved by the simple explicit finite difference algorithm. To simplify the formulation, we introduce the standard finite difference notations, such as
the forward differences:
and the backward differences:
We note that it is important to evaluate the nonlinear term, which we denote as
in (
For all
Then we compute the curvature projection on the wavelet basis by
The method proposed in this section is based on removing redundancy at the encoder and restoring the removed information using an inpainting method at the decoder. In this algorithm, redundancy removal is performed through detecting texture regions with similar statistical characteristics and dividing the image into homogeneous regions. The overall system with encoder and decoder diagrams is depicted in Figures
Block diagram of proposed image compression scheme–encoder.
Block diagram of proposed image compression scheme–decoder.
The input image is subjected to image analysis by extracting edges from the images in order to identify both the structural and textural regions. An input image is divided into
The important blocks from both textural regions and structural regions are selected by using different algorithms [
Gradient descent back propagation algorithm [
Finally, a set of outputs is produced as the actual response of the network. During the forward pass, the synaptic weights of the networks are all fixed. During the back pass, the synaptic weights are all adjusted in accordance with an error correction rule. The actual response of the network is subtracted from the desired response to produce an error signal. This error signal is then propagated through the network against the direction of synaptic conditions. The synaptic weights are adjusted to make the actual response of the network move closer to the desired response.
(1) Divide the original image in frequency domain into 8 × 8 pixel blocks and reshape each one into 64 × 1 column vector. (2) Arrange the column vectors into a matrix of 64 × 1024. (3) Let the target matrix equal to the matrix in step 2. (4) Choose a Gradient Descent Back Propagation learning algorithm to start training. (5) Simulate the network with the input matrix and the target matrix. (6) Obtain the output matrices of the hidden layer and the output layer. (7) Post-process them to obtain the compressed image, and the reconstructed image, respectively.
After performing the ANN based compression at encoder, the decompression process is carried out at decoder. Final reconstructed image is obtained through
The continuous shearlet transform [
Each analyzing element
(a) The tiling of the frequency plane. The tiling of horizontal cone is illustrated in solid line; the tiling of vertical cone is in dashed line. (b) Frequency support of shearlet satisfies parabolic scaling (
By sampling the continuous shearlet transform, we can get a discrete transform which is shown in Figure
(1) Decompose applying Laplacian Pyramid Scheme. (2) Use pseudo polar grid to compute (3) Band pass filter the output matrix (4) Re-assemble the Cartesian sampled values (5) Take the 2D IFFT.
Succession of Laplacian pyramid and directional filtering.
From the pure inpainting perspective, the inpainting problem may be stated as follows. Let
(1) Obtain a noisy image as input (2) Smooth the noisy image using (3) Analyze the smoothed image by extracting edges through (4) Identify the structural and textural regions based on the extracted edges. (5) Skip the regions that are not selected as necessary information instead fill the DC value and create a DC filled image (6) Perform ANN Compression and Decompression using where (7) Start the inpainting process with initial guess
Where (8) While (a) Set (b) Calculate (c) For all (d) Compute error (e) End the while loop
(a) Illustration of the inpainting problem. (b) The simplified architecture for image coding with inpainting.
The missing texture blocks are filled in with the texture from its surrounding [
We illustrate the performance of the proposed algorithm for image compression with image inpainting in shearlet domain using ANNs and compare it with the image inpainting method proposed by Liu et al. [
The proposed algorithm is tested with color images from USC-SIPI and Kodak image database. In all testes, we use shearlet base
Figure
Comparison with JPEG 2000 with QP = 75. (a) Input image. (b) Edges. (c) Image with 25% of portions removed. (d) Reconstructed image after inpainting and texture synthesis. (e) Reconstructed image by JPEG 2000.
Based on the preserved blocks, the shearlet
Bit-rate saving of proposed scheme compared to JPEG 2000 and edge based image inpainting method (QP = 75).
Original image | Block removal (%) | PSNR (dB) | Bit-rate (bpp) | Bit-rate saving (%) | |||
---|---|---|---|---|---|---|---|
JPEG 2000 | Edge based image inpainting | Proposed scheme | Compared with JPEG 2000 | Compared with edge based Image inpainting | |||
Jet | 25.0 | 45.67 | 1.156 | 0.919 | 0.780 | 32.53 | 15.13 |
Lena | 25.0 | 37.23 | 1.112 | 0.888 | 0.710 | 36.15 | 20.05 |
Peppers | 32.5 | 39.87 | 1.217 | 0.965 | 0.812 | 33.28 | 15.85 |
Kodim02 | 51.0 | 35.78 | 1.058 | 0.709 | 0.529 | 50.00 | 25.39 |
Kodim03 | 42.8 | 36.45 | 0.895 | 0.608 | 0.485 | 45.81 | 20.23 |
Kodim07 | 32.8 | 39.34 | 1.079 | 0.802 | 0.622 | 42.35 | 22.44 |
Kodim11 | 35.0 | 38.00 | 1.368 | 1.047 | 0.716 | 47.66 | 31.61 |
Kodim19 | 26.7 | 41.34 | 1.276 | 0.915 | 0.680 | 46.71 | 25.68 |
Kodim20 | 54.2 | 33.11 | 0.897 | 0.638 | 0.522 | 41.81 | 18.18 |
Kodim23 | 53.0 | 34.12 | 0.821 | 0.567 | 0.492 | 40.07 | 13.23 |
Figure
Bit rate savings of proposed scheme compared to H.264/AVC intra and edge based image inpainitng method (QP = 24).
Original image | Block removal (%) | PSNR (dB) | Bit-rate (bpp) | Bit-rate saving (%) | |||
---|---|---|---|---|---|---|---|
H.264 | Edge based image inpainting | Proposed scheme | Compared with H.264 | Compared with edge based image inpainting | |||
Jet | 25.0 | 42.56 | 0.985 | 0.880 | 0.727 | 26.16 | 17.35 |
Lena | 40.0 | 35.67 | 0.993 | 0.869 | 0.772 | 22.26 | 11.16 |
Peppers | 32.5 | 36.45 | 1.311 | 1.080 | 0.818 | 37.61 | 24.27 |
Mandrill | 43.0 | 36.48 | 0.880 | 0.783 | 0.691 | 21.47 | 11.74 |
Kodim02 | 51.0 | 32.11 | 0.948 | 0.701 | 0.510 | 46.20 | 27.25 |
Kodim03 | 42.8 | 34.23 | 0.710 | 0.562 | 0.495 | 30.28 | 11.92 |
Kodim07 | 32.8 | 35.00 | 0.876 | 0.751 | 0.671 | 23.38 | 10.63 |
Kodim11 | 35.0 | 36.78 | 1.354 | 1.098 | 0.736 | 45.68 | 33.01 |
Kodim13 | 42.0 | 37.02 | 0.900 | 0.872 | 0.714 | 20.66 | 18.12 |
Kodim19 | 26.7 | 39.36 | 1.246 | 0.956 | 0.647 | 48.07 | 32.32 |
Kodim20 | 54.2 | 30.00 | 0.823 | 0.636 | 0.473 | 42.49 | 25.58 |
Comparison with H.264/AVC with QP = 24. (a) Jet (25% removal); (b) peppers (32.5% removal); (c) kodim05 (23% removal); (d) kodim02 (51% removal). The top row shows the reconstructed images by proposed scheme and the bottom row shows the reconstructed images by H.264/AVC Intracoding.
Figure
Comparison with JPEG 2000 with QP = 75. (a) Kodim07 (32.8% removal); (b) kodim23 (53% removal); (c) kodim03 (42.8% removal); (d) kodim20 (54.2% removal); (e) kodim11 (35% removal). The first row shows the restored images by proposed algorithm and the second row shows the restored images by JPEG 2000.
Figure
Objective quality comparison between proposed scheme, H.264/AVC Intra, and JPEG 2000 on some typical color images.
The reconstructed images with same visual quality regardless of large PSNR difference are shown in Figure
Subjective quality comparison between the proposed scheme and H.264/AVC Intra. QP is 24 for high quality. From top to bottom: mandrill (43% removal) and kodim13 (42% removal). From left to right: incomplete image with black blocks; reconstructed image by the proposed scheme; reconstructed image by H.264/AVC Intra. Please note that the proposed scheme reconstructs both highly textured images with 21.07% bit-rate saving. Please observe that in kodim13 (the first row) the mandrill eye can also be reconstructed on both sides.
The recent no-reference quality assessment method is adopted in this paper that is proposed in [
Quality assessment (by measuring the blocking artifacts by the method in [
For the experiment, Intel 2 GHz CPU is used. The shearlet domain
The proposed algorithm is simpler because of the presence of region removal and assistant information generation based on Discrete Shearlet Transform (DST).
In this paper, we develop an ANNs based image compression framework that adopts shearlet domain inpainting technique. In this proposed algorithm, the correlated regions are identified and removed automatically at the encoder. Then they are restored at the decoder by using inpainting scheme. The key techniques used for coding are gradient descent back propagation algorithm with adaptive learning rate and
Edge extraction can be flexible and adaptable for compression. Finding the regions that can be eliminated is considered to be an open problem and it seems that solving this problem will lead to increase in compression ratios and output quality.
The authors declare that there is no conflict of interests regarding the publication of this paper.