Superresolution of Images Using Area Preserving Geometric Evolution Laws

The goal of superresolution is to construct out of a given low-resolution image an image with higher resolution. Superresolution is a common problem in image processing and numerous algorithms have been proposed [1]. Most of these algorithms are based on interpolation and lead to an improvement of the image but also to imperfect reconstructions with false high-frequency components. For interpolation methods based on PDEs, which are mostly adapted from inpainting problems, we refer to [2–6]. These approaches try to reconstruct the geometric properties of the images by evolving their level curves. In order to maintain fidelity to the original low-resolution image, different approaches are used. A standard level set method to modify pixel intensity which is based on evolution of mean curvature flow ∂tφ = H‖∇φ‖, where φ is the intensity and H the mean curvature, as used for example, in [7, 8] for edge-preserving smoothing, noise removal, and other image enhancement, smoothes away feature objects, as all level curves are shortening and thus will not lead to appropriate superresolution images. In [6], this is circumvented by imposing constraints that preserve accuracy to the original image using unchanging anchor pixels defining explicit topology constraints. In [4], the problem is attacked from a different point of view, by constraining the mean curvature flow problem in order to preserve the area enclosed by each level curve. We will follow this approach and extend it to other geometric evolution problems of higher order. Higher-order PDEs have been applied to various problems in image processing, for example, [9–11]. They are, however, not yet been used for superresolution. Only in [12] a higherorder model based on a Cahn-Hilliard equation is applied to superresolution of binary images, especially text. The paper is organized as follows: in Section 2, we describe a level set formulation for area conserving mean curvature flow, surface diffusion, and Willmore flow. In Section 3, we discuss an appropriate discretization which conserves all level curves simultaneously. In Section 4, the algorithms are applied to test problems. Thereby we start from a given 256 × 256 image, lower the resolution, and use bicubic interpolation to create an image that is again 256 × 256. Our three area preserving evolution laws are applied to this image for a short time and the results are compared with the original image. We also apply the algorithms to text images which are only known at a low resolution. In Section 5, we draw conclusions.


Introduction
The goal of superresolution is to construct out of a given low-resolution image an image with higher resolution.Superresolution is a common problem in image processing and numerous algorithms have been proposed [1].Most of these algorithms are based on interpolation and lead to an improvement of the image but also to imperfect reconstructions with false high-frequency components.For interpolation methods based on PDEs, which are mostly adapted from inpainting problems, we refer to [2][3][4][5][6].These approaches try to reconstruct the geometric properties of the images by evolving their level curves.In order to maintain fidelity to the original low-resolution image, different approaches are used.A standard level set method to modify pixel intensity which is based on evolution of mean curvature flow ∂ t φ = H ∇φ , where φ is the intensity and H the mean curvature, as used for example, in [7,8] for edge-preserving smoothing, noise removal, and other image enhancement, smoothes away feature objects, as all level curves are shortening and thus will not lead to appropriate superresolution images.In [6], this is circumvented by imposing constraints that preserve accuracy to the original image using unchanging anchor pixels defining explicit topology constraints.In [4], the problem is attacked from a different point of view, by constraining the mean curvature flow problem in order to preserve the area enclosed by each level curve.We will follow this approach and extend it to other geometric evolution problems of higher order.Higher-order PDEs have been applied to various problems in image processing, for example, [9][10][11].They are, however, not yet been used for superresolution.Only in [12] a higherorder model based on a Cahn-Hilliard equation is applied to superresolution of binary images, especially text.
The paper is organized as follows: in Section 2, we describe a level set formulation for area conserving mean curvature flow, surface diffusion, and Willmore flow.In Section 3, we discuss an appropriate discretization which conserves all level curves simultaneously.In Section 4, the algorithms are applied to test problems.Thereby we start from a given 256 × 256 image, lower the resolution, and use bicubic interpolation to create an image that is again 256 × 256.Our three area preserving evolution laws are applied to this image for a short time and the results are compared with the original image.We also apply the algorithms to text images which are only known at a low resolution.In Section 5, we draw conclusions.

Level Set Method
For a given initial curve M 0 , a geometric evolution law defines a family of curves M(t), t ≥ 0 with M(0) = M 0 .If M(t) is described implicitly as a specific level curve of a function φ(t), we can evolve M(t) by evolving φ(t) by solving ∂ t φ + v ∇φ = 0, with v a given normal velocity.If v is determined through a geometric evolution law, we can implicitly evolve the curve M(t) by solving this level set equation [13].
To simultaneously evolve all level curves M c of a level set function φ, we can define a global energy using the coarea formula.Thereby, we consider with f a given energy density on the curve M c and an appropriate gradient flow with respect to a specific metric Following [14], we interpret φ as an element of the manifold L of level set ensembles and identify a tangent vector s = ∂ t φ on L with a motion velocity v of the corresponding level curve M c via the classical level set equation s + v ∇φ = 0.This allows to define the corresponding metric on L which reads for the L 2 metric and for the H −1 metric For both cases, we are now able to rewrite the simultaneous gradient flow of all level curves in terms of the level set function φ

Mean curvature flow
which is equivalent to for all functions η ∈ C ∞ 0 (Ω).From (7) the weak form can be obtained for various geometric evolution laws in level set form by choosing appropriate energy density f , see Table 1 (e.g., [14,15]).
To define area preserving versions of these geometric evolutions laws, we introduce a Lagrange multiplier λ c and define the Lagrangien with the area of the interior domain Ω c inside of M c .We thus obtain (10) with λ c uniquely determined by the constraint Mc ∂ t x ds = 0: The corresponding evolution laws read as follows: (i) area preserving mean curvature flow (ii) surface diffusion (which is already area preserving) with Δ M the surface Laplacian; (iii) area preserving Willmore flow thereby S denotes the shape operator on M c and • 2 the Frobenius norm.
To construct a level set formulation to simultaneously evolve all level curves M c under the constraint, that their enclosed area is preserved, we follow the same lines as for the unconstraint problem and define to obtain or equivalently for all functions η ∈ C ∞ 0 (Ω) and L = L φ .For area preserving mean curvature flow ( f = 1 and g φ = g L 2 φ ), we thus obtain The same formulation can be obtained by rewriting the formulation in [4] in weak form.For surface diffusion ( f = 1 and g φ = g H −1 φ ), we obtain with P = I − ∇φ/ ∇φ ⊗ ∇φ/ ∇φ the projection onto the tangent space T x M c defined for every x on M c .For area preserving Willmore flow ( f = h 2 and g φ = g L 2 φ ), we obtain with ω = − ∇φ h a weighted curvature.

Semi-Implicit Finite Element Discretization
We discretize the constraint evolution problems first in space using piecewise linear finite elements and then in time based on a semi-implicit backward Euler scheme.We thereby follow the discretizations of the unconstrained problems [11,14] and treat the Lagrange multiplier explicit by evaluating (with α = h, μ) for different discrete values of c and then interpolating to obtain λ, see [4].The algorithms are implemented in the adaptive finite element toolbox AMDiS [16].
We first demonstrate the smoothing and area preserving properties of the algorithms.We, therefore, evolve a perturbed circle.Thereby the radius of M 0 is initially defined as r = r 0 (1 + eps cos(freqθ)) with angle θ ∈ [0, 2π). Figure 1 shows the initial curve and the evolution of its averaged curvature over time towards the constant value h = 1/r 0 .We here only use the constraint for M 0 and let all other level lines evolve accordingly, thus only λ 0 is computed and λ = λ 0 .For all cases, the area remains constant; however, the smoothing differes from which a different behaviour if applied to images is expected.

Numerical Results
We start with a simple application, namely, the superresolution of text images.Therefor, we use the logo of our university and apply area preserving Willmore flow, mean curvature flow, and surface diffusion to smooth the jagged parts of the text given at a low resolution.Figure 2 shows the results and the improvement if compared with bicubic interpolation.The differences obtained with the three evolution laws are minor.
The described algorithms can be extended to color images by applying them separately to the individual color planes.As already mentioned in [6], the possible separation of such an approach, due to the independent evolution, is not of significant for the obtained results.
We consider two images, the "Lena" image and the "Monarch" image.For both images, originally given with a resolution 256 × 256, the resolution is reduced to 75 × 75 and bicubic interpolations is used to produce again a 256 × 256 image.We apply our three level set algorithms to the resulting images.Figure 3 shows results on the "Lena" image.
Applying the same conserved evolution laws to the "Monarch" image results in Figure 4.
In both cases, all area conserved evolution laws result in smoother contours, which is directly visible at the shoulder and the hat in the "Lena" image and along the white stripes in the "Monarch" image.The jagged parts remaining after bicubic interpolations are removed.Without the area constraint, the evolution would oversmooth the images and significant detail would be lost.The applied constraints help to maintain these details.The difference in the three evolution laws is most significant for mean curvature flow, which reduces the sharpness of the edges.The results for Willmore flow and surface diffusion are comparable.

Conclusion
We consider PDE bases approaches to reconstruct geometric properties of images by evolving their level curves.Applying these level set reconstructions to obtain superresolution images had already been shown to lead to improved results, as jagged parts of the image can be smoothed.However, unconstrained evolutions will lead to oversmoothing.The goal thus is to constrain the evolution laws such that visually significant properties remain.A simple attempt in this direction, which does not require image-specific treatments is to constrain the area of each level curve.We have shown that such constrained geometric evolution laws can improve results of other methods for image magnification.The resulting PDEs turn to become nonlocal which requires special treatment.For all cases of area conserved mean curvature flow, surface diffusion and area conserved Willmore flow we use a semi-implicit discretization in which the Lagrange multiplier is treated explicitly which drastically reduced the complexity of the discretization scheme of the higher-order PDEs.A comparison of the three evolution laws shows an improvement if fourth-order PDEs are used.Small-scale details seem to better remain than for the second-order PDE.The difference between area conserved Willmore flow and surface diffusion is only marginally in the considered examples.As surface diffusion is already area conserving and thus computationally less expensive, it should be the method of choice.

Figure 1 :
Figure 1: Area preserving evolution laws for perturbed circle with r 0 = 0.5, eps = 0.075, and freq = 16.0.Different time steps are used for the evolution laws.

Figure 2 :
Figure 2: Superresolution of text (a) bicubic interpolation, (b) Willmore flow, (c) mean curvature flow, (d) surfaces diffusion, (e) zoom from left to right: bicubic interpolation, Willmore flowm mean curvature flow, and surface diffusion.The evolution results are shown after 10 time steps.The number of time steps used results from our individual taste for the best result.

Figure 3 :Figure 4 :
Figure 3: "Lena" image from left to right: (a) bicubic interpolation, (b) Willmore flow, (c) mean curvature flow, (d) surfaces diffusion.The evolution results are shown after 20 time steps in the case of Willmore flow and surface diffusion and 8 time steps for mean curvature flow.The number of time steps used results from our individual taste for the best result.

Table 1 :
Construction of geometric evolution laws in level set form, with h the mean curvature.