VLSI Design

Volume 2013 (2013), Article ID 738057, 9 pages

http://dx.doi.org/10.1155/2013/738057

Research Article

Discrete Wavelet Transform on Color Picture Interpolation of Digital Still Camera

Yu-Cheng Fan and Yi-Feng Chiang

Department of Electronic Engineering and Graduate Institute of Computer and Communication Engineering, National Taipei University of Technology, 1, Section 3, Chung-Hsiao East Road, Taipei 10608, Taiwan

Received 5 October 2012; Accepted 18 January 2013

Academic Editor: Yeong-Kang Lai

Copyright © 2013 Yu-Cheng Fan and Yi-Feng Chiang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Many people use digital still cameras to take photographs in contemporary society. Significant amounts of digital information have led to the emergence of a digital era. Because of the small size and low cost of the product hardware, most image sensors use a color filter array to obtain image information. However, employing a color filter array results in the loss of image information; thus, a color interpolation technique must be employed to retrieve the original picture. Numerous researchers have developed interpolation algorithms in response to various image problems. The method proposed in this study involves integrating discrete wavelet transform (DWT) into the interpolation algorithm. The method was developed based on edge weight and partial gain characteristics and uses the basic wavelet function to enhance the edge performance and processes of the nearest or larger and smaller direction gradients. The experiment results were compared to those of other methods to verify that the proposed method can improve image quality.

1. Introduction

The basic principles of digital still cameras and traditional cameras are analogous. Traditional cameras use sensitization negatives to sense the input image. Digital still cameras project the input image onto a charge-coupled device (CCD), where it is transformed into a digital signal. The digital signal is then stored in a memory component after compression. However, this signal indicates the light intensity and not the color variation. Therefore, a color filter array must be employed for digital sampling. Color filter arrays typically employ the RGB original color separation technique, where red, green, and blue values are mixed into a complete color image after the original image is passed through three color filter arrays. Because of the high costs and large space required to use three color filter arrays with CCDs, only one color filter array with a CCD is employed. Consequently, each pixel possesses only one red, green, and blue color elements. The general color filter array in digital still cameras possesses a Bayer pattern [1], as shown in Figure 1. An interpolation algorithm must be employed to identify the two missing colors based on the surrounding pixels. The zipper effect or false colors are typically observed in images after interpolation. Numerous interpolation algorithms have been proposed to resolve these problems and obtain good image quality.

Figure 1: Bayer pattern color filter array.

Image interpolation methods possess spatial and frequency characteristics. Edge direction and nonedge direction interpolation methods adopt spatial characteristics. The adjacent pixels selected by nonedge direction interpolation methods are constant. Examples of this method type include the bilinear interpolation method [2] and color difference interpolation method [3]. Because these methods do not detect edges, the edges of partial images are blurred following interpolation. The adjacent pixels selected by edge direction interpolation methods are nonconstant. These methods can detect and reduce blurred edges in the horizontal and vertical directions of an image. Examples of this method type include the edge sensing interpolation method [4] and edge correlation sensing correction interpolation method [5]. Frequency characteristic interpolation methods include the alternating projections interpolation method [6] and novel frequency-domain interpolation method [7]. Interpolation methods of this type use high- or low-frequency correlation to improve image aliasing and contrived phenomena and can provide high-quality images. A number of studies have employed a combination of the described methods or have proposed methods that use a wavelet algorithm for the edge or frequency domains [8–10]. Common research techniques are based on the physical characteristics of interference [11, 12]. Furthermore, the method combines edge and frequency algorithms for interpolation [13–15]. In [12], good missing green samples were first obtained based on the variances of color differences along a correct edge direction. The red and blue components were then interpolated based on the interpolated green plane. The refinement scheme was employed to improve the interpolation performance. The method employed in [15] obtains luminance values at the green sample locations and preserves high-frequency information. An adaptive filter was used to estimate the luminance values of the red and blue samples. Then, the estimated full-resolution luminance was used to interpolate the red, green, and blue color components. These results indicate that many interpolation methods result in contrived colors or blurred edges because they cannot sensitively detect edges or perform appropriate color interpolation. Therefore, effective interpolation of the image edge cannot be achieved. In this study, the relationship between the surrounding interpolation pixel weights and discrete wavelet transform (DWT) was used to perform color interpolation and edge detection. The results were then compared with those reported by other studies using conventional methods.

2. Proposed Method and DWT

2.1. Discrete Wavelet Transform

DWT can use the basic wavelet function and scaling function to conduct decomposition and reconstruction of sampling signals. The basic function is used to detect detailed variations. The scaling function is used to approximate original signals, which can be denoted as

The basic wavelet function can be calculated from the scaling function. and are digital filter coefficients; their relationship is expressed as

In wavelet transform, and are approximately equal to a high-pass filter and a low-pass filter. In (2), denotes the filter length. DWT has a similar function as a filter and can analyze the signal layer by layer. This filter comprises a high-pass filter and a low-pass filter. Figure 2 shows the operational manner and first-order wavelet transform decomposition of this filter. is an approximate coefficient. This indicates that the signal passes through a low-pass filter and undergoes downsampling. Approximate coefficients retain low-frequency information of the original signal and less high-frequency noise. is a detailed coefficient. This indicates that the signal passes through a high-pass filter and undergoes downsampling. Detailed coefficients retain high-frequency information of the original signal . Figure 2 is used in 2 to denote downsampling, which involves retaining half low-frequency and half high-frequency data. The method involves sampling odd and even terms.

Figure 2: First-order wavelet transform decomposition.

The wavelet transform decomposition process is expressed as

2.2. Proposed Method

Image information is obtained after passing through a Bayer pattern color filter array. Horizontal and vertical direction information is used to interpolate the green portion. For the red and blue portions, only information in the horizontal, vertical, and diagonal directions can be employed, as shown in Figure 3. Therefore, in this study, the wavelet sensitivity and color correlation weight [4] are used to identify the horizontal, vertical, and/or diagonal directions and interpolate missing pixels.

(a)

(b)

Figure 3: (a) Green interpolation and (b) red or blue interpolation.

Please refer to Figure 4. The green interpolation method can be expressed asshown in Algorithm 1.

if (>0)&(>0)⋯

    &(<Thd)&(<Thd)⋯

    &(>0&>0&>0)

    cD1H=+;

    cD2V=+;

else

    [cA1, cD1]=DWT(G8, G18);

    [cA2, cD2]=DWT(G12, G14);

    cD1H=+;

    cD2V=+;

end

if (cD2V>cD1H)

    G13=(G8+G18)/2+(R13*2-R3-R23)/4;

elseif (cD1H>cD2V)

    G13=(G12+G14)/2+(R13*2-R11-R15)/4;

else

    G13=(G8+G18+G12+G14)/4;

end

Algorithm 1

Figure 4: Interpolation reference.

cD1H is the horizontal gradient. cD2V is the vertical gradient. Thd is determined through an experiment and is used to limit the range of the nearest or smaller direction gradients. When the horizontal or vertical gradients are small, DWT is used to detect edges and is judged according to gradients. Because of the good sensitivity of DWT, detailed coefficients cD1 and cD2 can be obtained to enhance the gradients. If an edge exists, it has the correct cD1 or cD2 to determine the edge direction. G8 and G18 are calculated by DWT, which produces cA1 and cD1. G12 and G14 follow the same procedure. Figure 5 shows the flowchart of the processes in Algorithm 1. First, the operations were assessed, the edge direction was determined, and then interpolation was conducted.

Figure 5: Flowchart of the processes in Algorithm 1.

Please refer to Figure 4. The red and blue interpolation methods can be expressed as shown in Algorithm 2.

if (<Thd1&<Thd1&<Thd1)

    [cA1, cD1]=DWT(B7, B19);

    [cA2, cD2]=DWT(B17, B9);

    cD1H=;

    cD2V=;

    if cD1H<cD2V

        B13=G13+(B7-G7+B19-G19)/2;

    elseif cD1H>cD2V

        B13=G13+(B9-G9+B17-G17)/2;

    else

        B13=G13+(B7-G7+B19-G19+B9-G9+B17-G17)/4;

    end

else

    B13=G13+(B7-G7+B19-G19+B9-G9+B17-G17)/4;

end

Algorithm 2

When the gradients are relatively close, DWT is used to detect edges; otherwise, the color correlations are employed to interpolate directly. Thd1 is determined through an experiment. Furthermore, the edge direction of large gradients can be adjusted to limit the range of the nearest or larger direction gradients. Figure 6 shows a flowchart of the processes in Algorithm 2. First, the operations are assessed. If DWT is employed, the edge direction is determined before conducting interpolation; otherwise, interpolation is directly performed. The horizontal and vertical directions only possess information of the two adjacent pixels; thus, their correlation is directly employed for interpolation. The interpolation method can be expressed as

Figure 6: Flowchart of the processes in Algorithm 2.

The green interpolation method is identical to the red and blue interpolation methods. Diagonal interpolation of red and blue follows the same method used for blue and red. Furthermore, the red horizontal or vertical interpolation method is identical to that of blue.

3. Simulation Result

This study employed 24 standard color pictures provided for popular use. These images measure or , as shown in Figure 7, and have been included in numerous studies. To begin the simulation, raw image data are read and then separated into red, blue, and green image planes before sampling using the Bayer pattern. Regarding wavelet function selection, for this study, we adopted Daubechies wavelets for the basic functions and used the db2 function for simulation. Db2 is a second-rank Daubechies wavelet with a filter length of 4 and four low-pass filter coefficients denoted as , , , and . The values of these coefficients are 0.34151, 0.59151, 0.15849, and −0.091506. Employing the db2 wavelet for simulation provides the optimum result. Db2 to db7 were analyzed using, which indicated that shorter filters provide superior results. Furthermore, only one transformation is required. When transformed numerous times, waveforms are stretched to the left and right. Figure 8 shows a db2 wavelet waveform. Wavelet waveforms and filter coefficients are closely linked. Employing the adaptive wavelet function to analyze signals provides superior results. The waveform compression and stretch characteristics provide superior results in the shortest time variations and can be understood through observation. According to the experiment results, Thd and Thd1 were 223 and 1, respectively. The experimental process is shown in Tables 1 and 2. Picture 01 was used as an example. Furthermore, 223 and 1 were the stable range values.

Table 1: Peak signal-to-noise ratio (unit: dB).

thd

(thd1 = 1)
R
G
B
Average

50
37.6492
39.2368
37.7998
38.2286

100
37.6681
39.2605
37.8147
38.2478

150
37.6753
39.2719
37.8249
38.2574

200
37.6745
39.2742
37.8265
38.2584

210
37.6745
39.2742
37.8265
38.2584

220
37.6745
39.2742
37.8265
38.2584

221
37.6745
39.2742
37.8265
38.2584

222
37.6745
39.2742
37.8265
38.2584

223
37.6745
39.2742
37.8265
38.2584

224
37.6745
39.2742
37.8265
38.2584

225
37.6745
39.2742
37.8265
38.2584

230
37.6730
39.2722
37.8267
38.2573

240
37.6730
39.2722
37.8267
38.2573

250
37.6730
39.2722
37.8267
38.2573

Table 2: Peak signal-to-noise ratio (unit: dB).

thd 1 

(thd = 223)
R
G
B
Average

1
37.6745
39.2742
37.8265
38.2584

2
37.6745
39.2742
37.8265
38.2584

3
37.6742
39.2742
37.8261
38.2582

4
37.6734
39.2742
37.8252
38.2576

5
37.6719
39.2742
37.8239
38.2567

10
37.6597
39.2742
37.8091
38.2477

20
37.6127
39.2742
37.7594
38.2154

30
37.5647
39.2742
37.7081
38.1823

40
37.5103
39.2742
37.6582
38.1476

50
37.4765
39.2742
37.6223
38.1243

100
37.4035
39.2742
37.5498
38.0758

150
37.3927
39.2742
37.5365
38.0678

200
37.3926
39.2742
37.5357
38.0675

250
37.3923
39.2742
37.5346
38.0670

Figure 7: Standard images.

Figure 8: db2 wavelet waveform.

Table 3 shows a comparison of the peak signal-to-noise ratio (PSNR) in this study with that of other studies. The organizational sequence for the standard images in Figure 7 was from left to right and top to bottom. These images were named according to a nominal scale as Picture 01 to Picture 24. After processing, the images differ from their original appearance. To examine the image quality, PSNR is typically contrasted. PSNR can be expressed as

of the mean square error (MSE) is the pixel value of the original image and is located at . is the pixel value of position after image processing. The unit of PSNR is in decibels (dB). A larger PSNR indicates less aliasing. denotes the largest image pixel color value. If 8 bits are used to represent each sample pixel, the total bit number is 255. The results in Table 3 show that the proposed method provides a superior image quality compared to that in previous studies. Most images have large low-frequency areas. Thus, the spatial domain interpolation algorithm can be used to process most of images. However, if images possess a large fluid wave, wood, or grassy area, similar to Picture 22, the frequency domain algorithm provides a superior performance. Table 4 shows a PSNR evaluation, including each RGB color component and their average.

Table 3: Peak signal-to-noise ratio comparison (unit: dB).

Studies
Sakamoto et al. [2]
Gunturk et al. [6]
Pei and Tam [3]
Dubois [7]
Lukac et al. [5]
Chung et al. [13]
Proposed

Picture 01
30.85
38.05
34.08
38.02
37.41
39.88
38.26

Picture 02
36.03
37.55
35.54
37.88
37.78
37.96
38.96

Picture 03
35.88
42.06
39.78
42.17
42.32
42.91
43.70

Picture 04
28.62
34.73
31.57
35.13
34.29
36.22
34.62

Picture 05
35.60
41.75
39.36
41.71
41.59
42.05
43.09

Picture 06
32.57
39.22
36.06
38.86
38.74
40.17
39.72

Picture 07
36.03
39.07
39.16
39.32
40.01
39.79
41.92

Picture 08
32.00
37.94
35.25
39.96
37.45
40.75
39.19

Picture 09
35.76
42.11
40.29
42.17
42.19
42.32
43.35

Picture 10
33.13
35.89
35.93
36.04
37.07
36.24
39.28

Picture 11
32.30
37.39
35.43
37.27
37.09
37.68
38.27

Picture 12
34.21
37.93
36.93
38.22
38.52
38.45
40.80

Picture 13
36.73
41.45
41.23
41.69
42.39
42.31
43.25

Picture 14
36.36
41.57
40.72
41.97
42.48
41.67
43.96

Picture 15
33.36
39.31
37.86
39.81
39.14
40.56
40.41

Picture 16
35.02
38.94
38.92
39.27
39.25
39.33
40.91

Picture 17
32.50
40.38
35.51
40.42
39.34
41.75
41.98

Picture 18
37.75
41.71
42.12
42.19
42.93
42.07
43.73

Picture 19
36.51
40.38
39.94
40.52
40.79
40.15
42.66

Picture 20
28.59
37.26
31.04
35.27
34.57
37.56
36.76

Picture 21
36.34
42.52
40.33
42.98
42.27
43.54
44.55

Picture 22
34.83
42.10
38.32
43.71
40.45
44.31
41.63

Picture 23
33.03
40.83
38.99
40.39
41.18
41.58
40.80

Picture 24
30.66
34.92
33.95
35.34
34.89
35.29
36.46

Average
33.94
39.38
37.43
39.60
39.34
40.19
40.76

Table 4: Peak signal-to-noise ratio (unit: dB).

R
G
B
Average

Picture 01
37.6746
39.2743
37.8266
38.2585

Picture 02
38.4116
39.9794
38.4853
38.9588

Picture 03
43.3799
45.1411
42.5881
43.7030

Picture 04
34.4073
35.2576
34.1865
34.6171

Picture 05
43.0901
44.0063
42.1758
43.0907

Picture 06
39.3456
40.6480
39.1629
39.7188

Picture 07
40.0361
44.1844
41.5367
41.9191

Picture 08
38.7940
40.3259
38.4501
39.1900

Picture 09
42.8313
44.7694
42.4572
43.3526

Picture 10
37.8463
41.1899
38.8066
39.2809

Picture 11
38.1024
39.1079
37.6140
38.2748

Picture 12
39.4213
42.0331
40.9348
40.7964

Picture 13
42.7354
45.0743
41.9345
43.2481

Picture 14
43.0495
45.4716
43.3727
43.9646

Picture 15
39.3880
41.5611
40.2815
40.4102

Picture 16
39.4140
42.9145
40.4108
40.9131

Picture 17
41.4268
42.9398
41.5827
41.9831

Picture 18
42.5319
45.8168
42.8554
43.7347

Picture 19
41.5696
44.2054
42.2152
42.6634

Picture 20
36.1300
37.8601
36.2973
36.7625

Picture 21
43.4639
45.9502
44.2462
44.5534

Picture 22
41.4209
43.0842
40.3746
41.6266

Picture 23
39.7628
42.0993
40.5352
40.7991

Picture 24
36.2665
37.6725
35.4316
36.4569

Average
40.02083
42.10696
40.15676
40.7615

Figures 9 and 10 show Pictures 1 and 6, respectively. These images were selected from among the 24 standard images. The proposed method and conventional interpolation methods were used for simulation. Magnifying the images shows that the proposed method improves image quality significantly, as shown in Figure 11.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 9: (a) Original picture, (b) bilinear interpolation, (c) color difference interpolation, (d) edge detection interpolation, (e) adaptive color plane interpolation, and (f) the proposed method.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 10: (a) Original picture, (b) bilinear interpolation, (c) color difference interpolation, (d) edge detection interpolation, (e) adaptive color plane interpolation, and (f) the proposed method.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 11: (a) Original picture, (b) bilinear interpolation, (c) color difference interpolation, (d) edge detection interpolation, (e) adaptive color plane interpolation, and (f) the proposed method.

The images processed by using the proposed method are extremely similar to the original images. Figure 11 shows a magnification of Figure 9. DWT exhibited good edge detection sensitivity and partially resolved the zipper effects, color shifts, aliasing artifacts, blur effects, and obvious unnatural color grains. Furthermore, DWT limited the unnatural colors of the window lattice and the color inaccuracies of the crisscross.

4. Conclusion

Science and technology change every day. Although chip processing speeds continue to accelerate, their size and costs are increasingly decreasing. The proposed method does not employ frequency characteristics; instead, image quality is enhanced using spatial characteristics. Previous studies have discussed the importance of edges and interpolation pixels and calculated the frequency and spatial characteristics. This study exploited the sensitivity of wavelet algorithms and the correlation between colors to obtain good results regarding image edges and interpolation pixels. Comparing the simulation results to those of previous studies, the experimental images and data indicate that the proposed method can provide high-quality images.

Conflict of Interests

The authors do not have a direct financial relation with the commercial identity (Kodak Company and MATLAB/TOOLBOX) mentioned in our paper that might lead to a conflict of interests for any of the authors.

Acknowledgments

This study was supported by the Taiwan e-learning and Digital Archives Program (TELDAP) and the National Science Council of Taiwan under Grant no. NSC 100-2631-H-027-003. The authors gratefully acknowledge the Chip Implementation Center (CIC) for supplying the technology models used for IC design.

References

B. E. Bayer, “Color imaging array,” U.S. Patent 3 971 065, 1976

T. Sakamoto, C. Nakanishi, and T. Hase, “Software pixel interpolation for digital still cameras suitable for a 32-bit MCU,” IEEE Transactions on Consumer Electronics, vol. 44, no. 4, pp. 1342–1352, 1998.

S. C. Pei and I. K. Tam, “Effective color interpolation in CCD color filter arrays using signal correlation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 6, pp. 503–513, 2003.

J. E. Adams Jr., “Design of practical color filter array interpolation algorithms for digital cameras,” in 2nd Real-Time Imaging, vol. 3028 of Proceedings of SPIE, pp. 117–125, February 1997.

R. Lukac, K. N. Plataniotis, D. Hatzinakos, and M. Aleksic, “A new CFA interpolation framework,” Signal Processing, vol. 86, no. 7, pp. 1559–1579, 2006.

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Transactions on Image Processing, vol. 11, no. 9, pp. 997–1013, 2002.

E. Dubois, “Frequency-domain methods for demosaicking of bayer-sampled color images,” IEEE Signal Processing Letters, vol. 12, no. 12, pp. 847–850, 2005.

B. G. Jeong, S. H. Hyun, and I. K. Eom, “Edge adaptive demosaicking in wavelet domain,” in Proceedings of the 9th International Conference on Signal Processing (ICSP'08), pp. 836–839, Beijing, China, October 2008.

L. Chen, K. H. Yap, and Y. He, “Color filter array demosaicking using wavelet-based subband synthesis,” in Proceedings of the IEEE International Conference on Image Processing (ICIP'05), pp. II-1002–II-1005, September 2005.

J. Driesen and P. Scheunders, “Wavelet-based color filter array demosaicking,” in Proceedings of the International Conference on Image Processing (ICIP'04), vol. 5, pp. 3311–3314, October 2004.

X. Wu and X. Zhang, “Joint color decrosstalk and demosaicking for CFA cameras,” IEEE Transactions on Image Processing, vol. 19, no. 12, pp. 3181–3189, 2010.

K. H. Chung and Y. H. Chan, “Color demosaicing using variance of color differences,” IEEE Transactions on Image Processing, vol. 15, no. 10, pp. 2944–2955, 2006.

K. L. Chung, W. J. Yang, W. M. Yan, and C. C. Wang, “Demosaicing of color filter array captured images using gradient edge detection masks and adaptive heterogeneity-projection,” IEEE Transactions on Image Processing, vol. 17, no. 12, pp. 2356–2367, 2008.

K. L. Chung, W. J. Yang, P. Y. Chen, W. M. Yan, and C. S. Fuh, “New joint demosaicing and zooming algorithm for color filter array,” IEEE Transactions on Consumer Electronics, vol. 55, no. 3, pp. 1477–1486, 2009.

N. X. Lian, L. Chang, Y. P. Tan, and V. Zagorodnov, “Adaptive filtering for color filter array demosaicking,” IEEE Transactions on Image Processing, vol. 16, no. 10, pp. 2515–2525, 2007.