Multiscale Image Representation and Texture Extraction Using Hierarchical Variational Decomposition

In order to achieve a mutiscale representation and texture extraction for textured image, a hierarchical (í µí°µí µí±, í µí°º í µí± , í µí°¿ 2) decomposition model is proposed in this paper. We firstly introduce the proposed model which is obtained by replacing the fixed scale parameter of the original (í µí°µí µí±, í µí°º í µí± , í µí°¿ 2) decomposition with a varying sequence. And then, the existence and convergence of the hierarchical decomposition are proved. Furthermore, we show the nontrivial property of this hierarchical decomposition. Finally, we introduce a simple numerical method for the hierarchical decomposition, which utilizes gradient decent for energy minimization and finite difference for the associated gradient flow equations. Numerical results show that the proposed hierarchical (í µí°µí µí±, í µí°º í µí± , í µí°¿ 2) decomposition is very appropriate for multiscale representation and texture extraction of textured image.

A celebrated decomposition easier to implement is the total variation (TV) minimization model by Rudin, Osher, and Fatemi (ROF) [3] for image denoising, in which an image  ∈  2 (Ω) is split into  ∈ (Ω) and V ∈  2 (Ω): which yields so-called (,  2 ) decomposition.This model is convex and easy to solve in practice.The function  ∈ (Ω) allows for discontinuities along curves; therefore, edges and contours are preserved in the restored image .However, as Meyer pointed out in [15], the function space  2 (Ω) is not the most suitable one to model oscillatory components, since the oscillatory functions do not have small  2 -norms.He suggested using ((Ω))  , the dual space of (Ω), instead of  2 (Ω) for the oscillatory components.However, there is no known integral representation of continuous linear functional on (Ω).To address this problem, Meyer used another slightly larger space to approximate ((Ω))  .Using (Ω) to characterize oscillatory components {|| (Ω) + ‖V‖   (Ω) ,  =  + V} .(3) In [6], Vese and Osher did not solve (3) directly but adapted the model by adding a fidelity term into the energy functional to guarantee  ≈  + V.In detail, their variational formulation is defined as inf ∈(Ω),V∈  (Ω) In this (,   ,  2 ) decomposition, the image  is discomposed into three components,  =  + V +  with  ∈ (Ω), V ∈   (Ω), and  ∈  2 (Ω).
The previous models are examples of a larger class of the fixed scale decompositions (the scale parameters in these models are fixed).It has been argued that a human visualizes a scene in multiple scales [16,17].Then, multiscale approaches are appropriate for image representation because a single scale may not be a perfect simulation of the human visual perception.In order to achieve reliable image information in different scales, both the large-scale and small-scale behaviors should be investigated and incorporated appropriately.Thus, a natural way to address this problem is the multiscale analysis.
Tadmor et al. [5,13] presented a hierarchical decomposition based on the ROF model (1) to achieve multiscale image representation, in which the scale parameter is not fixed, but a varying sequence: starting with an initial scale  0 , and then, successive application of the following dyadic refinement step produces, after  such steps, the hierarchical (,  2 ) decomposition of : In this study, we focus on multiscale representation and texture extraction for textured image.As discussed previously, the (,  2 ) decomposition is not the best one for textured image, so using hierarchical (,  2 ) decomposition (7) introduced by Tadmor et al. to implement multiscale representation and texture extraction for textured image is obviously not the best choice.We thus in this paper propose the hierarchical decomposition using the (,   ,  2 ) model ( 4), which enables us to capture an intermediate regularity between  2 (Ω) and (Ω) and oscillation between  2 (Ω) and   (Ω).We here adopt (,   ,  2 ) decomposition because   (Ω) is a very suitable function space to model oscillatory patterns [6,14]; in addition, the   -norm is easier to solve in practice.In the proposed hierarchical (,   ,  2 ) decomposition, the scale parameter is not fixed but varies over a sequence of dyadic scales.Consequently, the decomposition of a textured image is not predetermined but is resolved in terms of layers of intermediate scales.So, we can achieve multiscale image representation.Compared to Tadmor et al. 's 2-tuple hierarchical decomposition, the proposed 3-tuple hierarchical decomposition can precisely extract texture in different scales.

Preliminaries
So far, there have been a lot of efficient variational decomposition models for textured image, much of which follows Meyer's work.The (,   ,  2 ) decomposition introduced by Vese and Osher is the first one to practically solve the Meyer's (, ) model presented in (2), in which cartoon component is measured in (Ω) and texture component in   (Ω), instead of (Ω).We here recall the definition and some known results of (Ω), (Ω) and   (Ω), which are much related to our present study.

Definition 1.
Let Ω ⊂ R 2 be an open subset with Lipschitz boundary.Then, (Ω) is the subspace of  1 (Ω) such that the following quantity Theorems 3 and 4 show the compactness and lower semicontinuity of (Ω).
Theorem 3 (see [18]).If   is a uniformly bounded sequence in (Ω), then there exist a subsequence    and  in (Ω) such that    converge to  in the -weak * topology.
For instance, consider the sequence of one-dimensional functions V  () = cos() defined on Ω = [0, /2].Then, V  () =    (), where   () = (1/) sin() + .It is easy to check that (1) This simple example demonstrates that an oscillatory function has a small -norm as well as   -norm which both approach to zero as the frequency of oscillations increases, but importantly, not with a so small  2 -norm.So, -norm and   -norm are more suitable than  2 -norm to measure textures in image decomposition.In addition,   -norm is weaker than -norm.So using   -norm to measure oscillatory functions, we also can exactly capture the texture in the energy minimization process.
For the space   (Ω), we have the following results which will be used in what follows.

The Proposed Hierarchical Decomposition
3.1.Description of Hierarchical Decomposition.We firstly modify the original (,   ,  2 ) decomposition presented in (4) to a single parameter pattern with a constraint condition ∫ Ω  = ∫ Ω .The new decomposition is defined as Here, the constraint condition ensures that the sum of texture V and residual (noise)  =  −  − V has zero mean.In this study, the parameter  in ( 14) is viewed as a scale factor which can be used to measure the scale of the extracted cartoon, especially texture.If the  value is too small, then only the small scale feature (coarser texture) is allocated in V  , while most of the large scale feature (smoother texture) is swept into the residual component   =  − (  + V  ).If  is too large, however, all the textures are extracted indiscriminately, regardless of their distinct scales.
To achieve multiscale description of a textured image, we here propose a hierarchical decomposition based on (14), which enables us to effectively extract textures in different scales.
For a given scale , the minimizer of   (, ; , V) is interpreted as a decomposition,  =   + V  +   , such that V  captures textures in the scale , while the textures above  remain unresolved in   .The residual   still consists of significant textures when viewed under a larger scale than , say 2: with where V 2 captures textures in the scale 2, while the textures above 2 remain unresolved in  2 .The process of ( 15) can be continued to capture the missing large scale textures.
The proposed hierarchical decomposition can be stated as follows: starting with an initial scale  =  0 , where Proceeding with successive applications of the dyadic refinement step (15), we have where From (19), we obtain, after  such steps, the hierarchical decomposition of  as follows: The partial sum, ∑  =0 (  + V  ), provides a multiscale representation of , in which ∑  =0   lies in the intermediate scale spaces between  2 and , and ∑  =0 V  lies in the intermediate scale spaces between   and  2 .Another application of this hierarchical decomposition is multiscale texture extraction.Indeed, ∑  =0 V  represents the textures in the scales ranging from  0 to  0 2  .

Existence of Hierarchical Decomposition.
The existence of our hierarchical decomposition is directly derived from the following result, actually, which can be used for original (,   ,  2 ) decomposition by replacing   with , but Vese and Osher did not give proof for it in their papers.
has a solution (, V) such that  ∈  and V ∈   .
Since   is uniformly bounded in  2 , by (23) we have that V  is uniformly bounded in  2 .Therefore, there exists V ∈  2 such that (up to a subsequence) V  converges to V weakly in  2 .By weak lower semicontinuity of  2 -norm, we deduce the following property: For V  ∈   , by Proposition 7, there exists , such that, up to a subsequence,  , con- Taking  → ∞ (using weak  2 topology and weak *   topology), we obtain This implies V = div(g) ∈ D  .And since V ∈  2 , V = div(g) a.e.Therefore, V ∈   ∩  2 .By weak * lower semicontinuity, it follows that By ( 26)-(30), we have which implies that (, V) is a solution for (22).The proof is completed.
Firstly, similar to (but slightly different from) Definition 5.3 of [8], we here define a new quantity ‖ ⋅ ‖ * to measure the  2 -function, which will play a key role in our following study.
By Theorem 9, the minimization problem ( 22) must have solutions.Next, simulating hierarchical (,  2 ) decomposition proposed by Tadmor et al. [5], we show some properties for these solutions, which will be used to demonstrate the nontrivial property for our hierarchical decomposition.
and that is, By the definition of ‖ ⋅ ‖ * , we have Proof.The first assertion is proved directly by Lemma 12.
Remark 15.By Theorems 9 and 14, we can deduce that for any  2 -function  with ∫ Ω  x ̸ = 0, there must be a nontrivial hierarchical decomposition.This result is much significant for image hierarchical decomposition.In general, a digital image  is a nonnegative  2 -function with ∫ Ω  x ̸ = 0, so any hierarchical (,   ,  2 ) decomposition of  must be nontrivial.

Convergence of Hierarchical Decomposition.
For the hierarchical decomposition given in ( 21), we have the following convergence result (Theorem 17) in the  2 topology, which is similar to the convergence result of hierarchical (,  2 ) decomposition proposed by Tadmor, Nezzar, and Vese (see Theorem 2.2 in [5] for details).To prove Theorem 17, we need the following lemma.
Next, we prove the second assertion that is, (57).Since Since summing up both sides of (63), we obtain Equation ( 57) can be seen as the  2 -energy decomposition of  in our hierarchical decomposition.In addition, the multiscale nature of our hierarchical extraction can be quantified in terms of this energy decomposition.

Numerical Implementation
In this section, we present the details of numerical implementation for our hierarchical (,   ,  2 ) decomposition: Taking V = div(g) = div( 1 ,  2 ), we obtain the following equivalent formulation of (67) in terms of ,  1 , and  2 : where Minimizing the energy in (68) with respect to ,  1 and  2 yields the following Euler-Lagrange equations: If the exterior normal to the boundary Ω is denoted by (  ,   ), then the associated boundary conditions for ,  1 , and  2 are Equation ( 69) with boundary condition (72) implies that ∫ Ω  = ∫ Ω   holds.Indeed, by taking the integral for each side of (69) and using the Gaussian formula, we obtain Since V = div(g) ∈   , by Proposition 8, we have ∫ Ω div(g) = 0. Therefore, ∫ Ω  = ∫ Ω   .We solve (69)-( 71) by the alternating algorithm.For each equation, we adopt gradient decent method.To simplify the presentation, we introduce the notation u 0 0 + 100 r 0 + 100 ∑  The details are as follows: (i) fixed ( 1 ,  2 ), find the solution  of with the initial condition (, , 0) =   (, ), (80) Then, (77)-( 78) can be approximated by the following discretizations (to remove the singularity when |∇| = 0 and √ 2 1 +  2 2 = 0, we introduce a regularity parameter  2 ): with the initial condition where   , is the curvature of the level set of  at the grid (  ,   , Δ), defined by with the initial condition

Numerical Results
We present four numerical examples in this section to demonstrate the efficiency of multiscale texture extraction and image representation using the proposed hierarchical (,   ,  2 ) decomposition for textured images.Test images, shown in Figure 1, are two synthetic images and two real images.In all experiments, we take the time step Δ = 0.05, the space step Δ = Δ = 1, the initial scale  0 = 0.005, and the regular parameter  2 = 10 −9 .
For the choice of , by the theoretical analysis in Section 2, we have that   -norms are weaker than -norm for any 1 ≤  < ∞.So, any choice of  with 1 ≤  < ∞ is suitable.Here, similar to what was done by Vese and Osher in [6,14], we tested the model (68) with different values of ; our observation is that results are very similar, while the case of  = 1 yields faster calculations per iteration.Thus, we set  = 1 in the following.We note in passing that some different approaches based on duality principle have been proposed, such as [21,22], to solve (67) with  = ∞.We here adopt the method introduced by Vese and Osher because this study is following their work in [6,14].
(i) Image hierarchical (,   ,  2 ) decomposition: Figure 2 shows the hierarchical decomposition results for a synthetic textured image for 7 steps.The first column shows the cartoon components of the initial image in different scales.We can see that these cartoon components are very little different visually.This phenomenon is compatible with the theory of causality of scale space.The second column shows the "textures+100" (plus a constant for illustration purposes) of the image in different scales.It is clear that the textures can be gently extracted by increasing the value of scale parameter  0 2 +1 , because this image involves the textures of different scales: coarser textures correspond to the smaller scales, while smoother textures correspond to the larger scales.The third column shows "residuals+100, " from which we can clearly see that some textures and edges are swept into these residual components when the value of scale parameter  0 2 +1 is smaller, and then, they are gradually swept out and absorbed by   and V  by increasing the value of the scale parameter  0 2 +1 .Figure 3 shows the plots of the -energy of   ,  energy of V  , and  2 -energy of   , respectively.
(ii) Multiscale texture extraction: Figure 4 shows the results of multiscale texture extraction using hierarchical decomposition for another synthetic textured image for 9 steps.The first two images show the initial and final cartoon components which have little visual difference; this phenomenon is identical with the results of the first experiment.The next nine images show the texture components in different   (iv) Multiscale image representation for noisy textured image: Figure 6 shows the hierarchical decomposition results of a noisy Barbana for 6 steps.The last column of this figure shows ∑  =0 (  + V  )s which can be seen as restored images in different scales.Clearly, when the value of  is smaller, such as  = 0,1, there are a few textures and noises in the restored images, much of which is swept into residual components.When  = 2, 3, some textures of the image are recovered on the headscarf of Barbana while removing the smaller scale noises from the entire image.If we continue the u 0 0 + 100 r 0 + 100 ∑  decomposition into smaller scales, then noise will reappear in the ∑  =0 (  + V  ) components, since the refined scales reach the same scales of the noise itself.From the last column of this figure, we can obtain restored image from noisy Barbana in different scales according to our requirements.

Conclusions
In this paper, in order to achieve multiscale image representation and texture extraction for textured image, we presented a hierarchical (,   ,  2 ) decomposition model which combines the idea of hierarchical decomposition introduced by Tadmor et al. with the (,   ,  2 ) decomposition proposed by Vese et al.In addition, we proved the existence and the convergence of the hierarchical decomposition, and the nontrivial property of this decomposition is also discussed.But the uniqueness of this hierarchical decomposition has not been proved in this paper.The authors will be concerned about this problem in the successive research.

Figure 1 :
Figure 1: Test images.Left to right: (a) and (b) two synthetic textured images; (c) fingerprint image; (d) a portion of noisy Barbana image which generated by adding Gaussian noise with standard deviation 20 to the clean data.

Figure 2 :
Figure 2: Hierarchical decomposition of a synthetic image for 7 steps.

Figure 3 :
Figure 3: Energy plots of three components.(a) The -energy of   .(b) The   -energy of V  .(c) The  2 -energy of   .

Figure 4 :
Figure 4: Multiscale texture extraction using hierarchical decomposition of a synthetic image for 9 steps.

Figure 5 :
Figure 5: Multiscale texture extraction and image representation using hierarchical decomposition of a fingerprint for 6 steps.

Figure 6 :
Figure 6: Multiscale image representation using hierarchical decomposition of a noisy Barbana for 6 steps.