Journal of Nanomaterials

Volume 2014 (2014), Article ID 931616, 6 pages

http://dx.doi.org/10.1155/2014/931616

Research Article

A New Method for Superresolution Image Reconstruction Based on Surveying Adjustment

Jianjun Zhu,¹ Cui Zhou,1,2 Donghao Fan,¹ and Jinghong Zhou¹

¹School of Geosciences and Info-Physics, Central South University, Changsha 410083, China

²College of Traffic Information, Hunan Communication Polytechnic, Changsha, Hunan 410132, China

Received 24 April 2014; Accepted 24 May 2014; Published 9 June 2014

Academic Editor: Yongfeng Luo

Copyright © 2014 Jianjun Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

A new method for superresolution image reconstruction based on surveying adjustment method is described in this paper. The main idea of such new method is that a sequence of low-resolution images are taken firstly as observations, and then observation equations are established for the superresolution image reconstruction. The gray function of the object surface can be found by using surveying adjustment method from the observation equations. High-resolution pixel value of the corresponding area can be calculated by using the gray function. The results show that the proposed algorithm converges much faster than that of conventional superresolution image reconstruction method. By using the new method, the visual feeling of reconstructed image can be greatly improved compared to that of iterative back projection algorithm, and its peak signal-to-noise ratio can also be improved by nearly 1 dB higher than the projection onto convex sets algorithm. Furthermore, this method can successfully avoid the ill-posed problems in reconstruction process.

1. Introduction

Obtaining images is a process from ideal high-resolution (HR) image to observation image. Superresolution image reconstruction (SRIR) is a reverse process of obtaining images, which uses many low-resolution (LR) images obtained by multiple observations on the same target to calculate the real image of the target. SRIR was firstly proposed by Harris [1] and Goodman [2]. It is similar to the process of surveying adjustment, which gets optimal value by multiobservations on the same object in field of surveying. SRIR can obtain one or more HR images from multiple low-resolution images with complementary information. It can overcome the limitation of hardware and correspondingly increase the image resolution. Nowadays, SRIR techniques play an important role in remote sensing, video security monitoring, military aerial reconnaissance, medical digital image processing, and video standard conversion. One of the most typical applications of SRIR technology is in SPOT5 satellite of France. The satellite takes two images with resolution of 5 m, but with mutual dislocation of 0.5 pixel. Images with resolution of 2.5 m are obtained by SRIR and provided to the whole world.

Up to now, a lot of algorithms have been proposed [3–16]. These algorithms can be roughly divided into two categories: one is frequency-domain algorithm [17–19] and the other is spatial-domain algorithm [4, 13]. In frequency-domain algorithms, images are transformed onto frequency domain and SRIR is carried in frequency domain. In spatial-domain algorithm, image gray value are computed directly in the process of SRIR. Frequency-domain algorithms are usually based on idealistic assumptions, so their application is very limited and cannot be effectively applied in most cases. While spatial-domain algorithm [20–25], such as nonuniform spatial interpolation, iterative back projection (IBP), has difficulty to make use of a priori information, hence the quality of SRIR will be severely limited [26]. Even the projection onto convex sets (POCS) algorithm, a real popular method, sometimes is unstable in computation and heavily depends on the initial estimation of SRIR, and its solution is often not unique and so on. Actually, most of the above two kinds of algorithms are proposed according to some specific situations; the performance of SRIR from multiple images usually depends on the imaging system and the accurate degradation model. The algorithm universality of SRIR is rarely dealt with. In surveying and mapping field, if the same object is observed multiple times, the optimum value of the object can be obtained by using surveying adjustment. SRIR is similar to this process. If a real image can be represented by a gray function, any pixel value will be the integration of its gray function in the pixel, and hence the relationship between observation images (namely, LR images in SRIR) and real image (namely, HR image in SRIR) can be described by an integration relationship. Based on this principle, a general surveying adjustment model containing integration is established in this paper. And, on this model, a new method for SRIR is proposed. The main idea of this new method for SRIR is that a sequence of low-resolution images are taken firstly as observations, and then observation equations are established by using survey adjustment method. The gray function of the real image is estimated on the observation equations. After that, the estimated gray function is used to calculate HR pixel value of the corresponding area. Finally, HR image can be obtained.

In order to evaluate quantitatively reconstruction image, peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are usually used as two evaluation indicators. PSNR describe gray similarity between two images, and SSIM describe structural similarity of two images. PSNR and SSIM are simultaneously used to get a comprehensive evaluation for reconstruction images in this paper [6].

2. The Principle of SRIR Based on the Surveying Adjustment

Pixel value of any image can be regarded as integration of its gray function within the scope of the pixel; namely,

where is pixel value, and are pixel coordinates, is pixel scope, and is the gray function of object surface. If of the real image is known, superresolution image with any resolution can be calculated by (1). Conversely, if gray values of pixels are known, we can estimate on these gray values of pixels. Basic idea is to take pixel gray values of LR images as observations and real value of gray function as unknowns and then use surveying adjustment method to estimate , which will be the closest to the real value of gray function, and then pixel values of SR can be calculated by gray function .

For one or more LR images coexist within the same area, it can be regarded as multiple observation for the real gray function in such area. Considering the errors which possibly existing in the different pixel gray values, the observation equations can be approximately expressed as follows:

where

The above observation equations can also be expressed as a matrix:

where is the vector of observation, is correction vector of observations, gray function is unknown, and is the number of pixels of LR images related to the same small region. It is an integral relationship between observations and unknowns in the above observation equations; thus we call (4) as “adjustment model with integration.” Different from traditional adjustment model, the functions here must be determined rather than parameters. However, how to find the solution of (4) needs to be further studied. For images, gray function can be denoted by Fourier series or by wavelet function. Here, we use polynomial to substitute the gray function and transform (4) into a model of parameters estimation. If gray function is replaced by quadratic polynomial within a sufficiently small region, such as

Substituting (5) into (2), the following is obtained:

where

Equation (4) can be written again as follows:

According to the least square adjustment method, the solution can be found as follows:

where is the weight of observations. Estimation of real gray function can be calculated by putting into (5), and then (10) is obtained by putting into (1):

According to (10), we can calculate pixel values of HR image in SRIR.

3. The Application of SRIR Based on Surveying Adjustment

When using (4) or (8) for SRIR, one key is to determine the region that the gray function will deal with, namely, how to select pixel values as observations. Generally, real gray function of object surface is very complicated; if we use a quadratic function to substitute the real gray function, it must be in a small region. In this paper, we determine the region according to the following method: for any pixel of the reference image, one can get its surrounding 8 pixels, and then total of 9 pixels can be taken as observations. For the second image after registration, one should select the pixel closest to pixel in reference image and its surrounding 8 pixels as observations, and so on. For every LR image, 9 pixels can be selected as observations. Based on these observations, an estimate of real gray function can be obtained according to (9). And then pixel value of coordinate and the surrounding 8 pixel values of SR image can be calculated by (10). Obviously, the computation of this method will converge and can successfully avoid the ill-posed problems.

When (9) is used, it is necessary to determine the weight of observations. In fact, pixel value above contains two kinds of noise: one is the noise of image itself and the other is the model error brought in by the substitution of quadratic polynomial. The model errors will depend on the distance to the center of the region using quadratic polynomial. Accordingly, the scheme of weighting observations is taken as follows: for any pixel of the reference image, one can calculate the distances between and the surrounding 8 pixels and the distances between and the corresponding 9 pixels of the second image. Then, the weight values of 17 observations are defined by inverse distances square; that is,

where denotes the distance between the concerned pixel and pixel and denotes the weight of the observation corresponding to the concerned pixel. The weight value of pixel itself will use

where is half pixel width.

Pixel values of HR image exist overlapping area according to (10); overlapping area can be shown in Figure 1. For the overlapping region, we use the average value as the last HR pixel value.

Figure 1: Schematic diagram of overlapping of HR pixels.

4. Testing and Results Analysis

In order to check the feasibility of the suggested method, we design tests according to the following idea: we add some noise to a HR image and then take samples from the HR image to obtain a series of LR images. By using these LR images, we try to reconstruct the HR image. Comparing with the original image, we can know the feasibility of the suggested method. The whole process can be divided into the following four steps: obtaining LR images, image registration, fuzzy estimation, and image reconstruction.

4.1. Obtaining LR Images

We can firstly select a HR image, and then we transform the HR image into a sequence of LR images. Image transformation includes image translation and rotation, fuzzy processing, down sampling, and image noise. An example with a size and gray-scale level 256 image is as shown in Figure 2; we simulate four LR images with subpixel displacement. The sequence of LR images is created through the following steps. Firstly, original HR image is shifted with different motion vectors in vertical and horizontal directions to produce four images. Secondly, these images are converted to frequency domain for setting frequency range 0-0.1 to zero; we can get four blurred images. Thirdly, these images are downsampled by the factor of 2 in each direction to produce four LR images of a size of . Finally, we add Gaussian noise to downsampled images, where the variance is 0.05 and the mean value is 0.

Figure 2: The desired HR image.

4.2. Image Registration

Image registration is to choose one LR image as reference frame, and the other LR images are aligned with reference frame on space. LR images registration is irregular sampling of HR grids. Registration models usually include translation model, affine transformation model, bilinear model, projection model, and rigorous model [26]. The scale invariant feature transform (SIFT) algorithm is adopted here to get subpixel precision in image registration. Image registration based on characteristics can be generally divided into the following three steps: feature extraction, feature description, and feature matching [27].

4.3. The Fuzzy Function and Noise Estimation

Fuzzy function describes image formation process affected by fuzzy degradation. The precision of the estimation of fuzzy function can directly affect the quality of SRIR. In these experiments, Gaussian fuzzy function is adopted. And Gaussian noise with mean 0 and variance 0.05 is added into LR image. Wiener filtering is used to deblur and denoise for LR images in the image preprocessing stage.

4.4. Image Reconstruction

A sequence of LR images are used to reconstruct the HR image with the same size of original image by IBP, POCS, and this algorithm. Reconstruction results are shown in Figure 3. We can see that the visual feeling of Figures 3(b) and 3(d) is better than that of Figure 3(c). For the detailed comparison, we can take a small part of the images and enlarge it in Figure 4. From Figure 4, we can clearly see that the reconstruction images have richer details. Finally, the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are both used to evaluate the objective reconstruction. PSNR is a measure of gray similarity between two images. SSIM is a measure of structural similarity of two images. PSNR and SSIM can be simultaneously used to evaluate comprehensively reconstruction image from gray similarity and structural similarity [24]. The results are shown in Table 1. One can see that PSNR of the suggested method is nearly 1 dB higher than that of the other two algorithms in Table 1. This means that reconstruction image by the suggested algorithm has higher gray level similarity with original image than the other two methods. From the value of SSIM, one can see that reconstruction image from the suggested algorithm in this paper has the same structural similarity with the other two algorithms.

Table 1: Evaluation results of PSNR and SSIM.

Evaluation index
IBP
POCS
Our algorithm

PSNR/dB
68.6987
71.5415
72.3819

SSIM
0.8324
0.8016
0.8762

(a)

(b)

(c)

(d)

Figure 3: The results of image reconstruction. (a) LR image, (b) image of IBP reconstruction, (c) image of POCS reconstruction, and (d) image of our algorithm reconstruction.

(a)

(b)

(c)

(d)

Figure 4: The results of intercepting part reconstruction image. (a) LR image, (b) image of IBP reconstruction, (c) image of POCS reconstruction, and (d) image of our algorithm reconstruction.

5. Conclusions

The method proposed in this paper is based on surveying adjustment theory. Examples show that the method is feasible. The quadratic polynomial can be used as gray function of object surface and the convergence speed of the methods is faster than that of POCS and IBP algorithm. Results also showed that the proposed method can obtain a good result of reconstruction image through the subjective and objective comprehensive evaluation.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to thank the National Basic Research Program (no. 2013CC733303), the National Natural Science Foundation of China (nos. 41274010 and 40974007), the National High Technology Research and Development Program of China (no. 2012AA121301), and the Fundamental Research Funds of Central South University for financial support.

References

J. L. Harris, “Diffraction and resolving power,” Journal of the Optical Society of America, vol. 54, no. 7, pp. 931–933, 1964.

J. Goodman, Introduction To Fourier Optics, McGaw-Hill Physical and Quantum Electronics Series, McGraw-Hill, New York, NY, USA, 1968.

M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Transactions on Image Processing, vol. 10, no. 8, pp. 1187–1193, 2001.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21–36, 2003.

M.-G. Hu, J.-F. Wang, and Y. Ge, “Super-resolution reconstruction of remote sensing images using multifractal analysis,” Sensors, vol. 9, no. 11, pp. 8669–8683, 2009.

L. Zhang, H. Zhang, H. Shen, and P. Li, “A super-resolution reconstruction algorithm for surveillance images,” Signal Processing, vol. 90, no. 3, pp. 848–859, 2010.

X. Li, Y. Hu, X. Gao, D. Tao, and B. Ning, “A multi-frame image super-resolution method,” Signal Processing, vol. 90, no. 2, pp. 405–414, 2010.

J. Tian and K.-K. Ma, “Stochastic super-resolution image reconstruction,” Journal of Visual Communication and Image Representation, vol. 21, no. 3, pp. 232–244, 2010.

H. Nasir, V. Stanković, and S. Marshall, “Singular value decomposition based fusion for super-resolution image reconstruction,” Signal Processing: Image Communication, vol. 27, no. 2, pp. 180–191, 2012.

N. Pinto, M. Simard, and R. Dubayah, “Using InSAR coherence to map stand age in a boreal forest,” Remote Sensing, vol. 5, no. 1, pp. 42–56, 2013.

H. Zhang, L. Zhang, and H. Shen, “A super-resolution reconstruction algorithm for hyperspectral images,” Signal Processing, vol. 92, no. 9, pp. 2082–2096, 2012.

M. Debella-Gilo and A. Kääb, “Measurement of surface displacement and deformation of mass movements using least squares matching of repeat high resolution satellite and aerial images,” Remote Sensing, vol. 4, no. 1, pp. 43–67, 2012.

M. Liu, J. Huang, M. Gao, and S. Qin, High Performance Super-Resolution Reconstruction of Multiple Images Based on Fast Registration and Edge Enhancement, Intelligence Science and Big Data Engineering, Springer, 2013.

A. G. Devi, T. Madhu, and K. L. Kishore, “An improved super resolution image reconstruction using SVD based fusion and blind deconvolution techniques,” International Journal of Signal Processing, Image Processing Pattern Recognition, vol. 7, no. 1, p. 283, 2014.

H.-X. Wang, Z.-M. Lu, Y. Zhang, and Z. Diao, “Sub-dictionary based sparse representation for efficient super-resolution image reconstruction,” Information Technology Journal, vol. 13, no. 1, pp. 94–101, 2014.

H. Zhang, Z. Yang, L. Zhang, and H. Shen, “Super-resolution reconstruction for multi-angle remote sensing images considering resolution differences,” Remote Sensing, vol. 6, no. 1, pp. 637–657, 2014.

A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '92), pp. 169–172, 1992.

C. E. Davila, “Efficient recursive total least squares algorithm for FIR adaptive filtering,” IEEE Transactions on Signal Processing, vol. 42, no. 2, pp. 268–280, 1994.

E. A. Kaltenbacher and R. C. Hardie, “High-resolution infrared image reconstruction using multiple low-resolution aliased frames,” in Proceedings of the IEEE Aerospace and Electronics Conference (NAECON '96), pp. 142–152, April 1996.

J. J. Clark, M. R. Palmer, and P. D. Lawrence, “A transformation method for the reconstruction of functions from nonuniformly spaced samples,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, no. 5, pp. 1151–1165, 1985.

H. Stark and P. Oskoui, “High-resolution image recovery from image-plane arrays, using convex projections,” Journal of the Optical Society of America. A, Optics and image science, vol. 6, no. 11, pp. 1715–1726, 1989.

M. Irani and S. Peleg, “Improving resolution by image registration,” Graphical Models and Image Processing, vol. 53, no. 3, pp. 231–239, 1991.

R. R. Schultz and R. L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Transactions on Image Processing, vol. 5, no. 6, pp. 996–1011, 1996.

M. Elad and A. Feuer, “Superresolution restoration of an image sequence: adaptive filtering approach,” IEEE Transactions on Image Processing, vol. 8, no. 3, pp. 387–395, 1999.

Z. Xinming and S. Lansun, “The Development of Super-Resolution Restoration from Image Sequences,” Measurement & Control Technology, vol. 21, no. 5, pp. 33–35, 2002.

A. G. Devi, T. Madhu, and K. L. Kishore, “An improved super resolution image reconstruction using SVD based fusion and blind deconvolution techniques,” International Journal of Signal Processing, Image Processing & Pattern Recognition, vol. 7, no. 1, pp. 283–297, 2014.

Y.-N. Kang, H. Huang, Y.-Y. Zhu, and P.-J. Lai, “Super-resolution image registration based on SIFT and its realization with MATLAB,” Computer Knowledge and Technology, vol. 5, no. 58, pp. 8031–8033, 2009.