A New Variational Model for Segmenting Objects of Interest from Color Images

We propose a new variational model for segmenting objects of interest from color images. This model is inspired by the geodesic active contour model, the region-scalable fitting model, the weighted bounded variation model and the active contour models based on the Mumford-Shah model. In order to segment desired objects in color images, the energy functional in our model includes a discrimination function that determines whether an image pixel belongs to the desired objects or not. Compared with other active contour models, our new model cannot only avoid the usual drawback in the level set approach but also detect the objects of interest accurately.Moreover, we investigate the new model mathematically and establish the existence of the minimum to the new energy functional. Finally, numerical results show the effectiveness of our proposed model.


Introduction
Image segmentation is one of the fundamental problems in image processing and computer vision.Recently, variational methods have been extensively studied for image segmentation because of their flexibility in modeling and advantages in numerical implementation.In generally these methods can be categorized into two classes: edge-based models 1-3 and region-based models 4-6 .
Edge-based models utilize the image gradient to stop evolving contours on object boundaries.The geodesic active contour GAC model is one of the most well-known edgebased models.It was proposed in 2 and widely used in practice 7 .However, this model has one major disadvantage: given an initial curve, during the evolution, the energy may evolve to its bad local minimizers.Region-based models have some advantages over the edge-based models.Firstly, region-based models do not utilize the image gradient, so they have better performance for images with weak object boundaries.Secondly, these models are significantly less sensitive to the location of initial contours.The Chan-Vese CV model 5 is one of the most popular region-based models.It is successful for an image with two regions, each of which has a distinct mean of pixel intensities.In order to handle images with multiple regions, Vese and Chan proposed the piecewise constant PC models 8 , in which multiple regions can be represented by multiple level set functions.And yet, these PC models are not very successful for images with intensity inhomogeneity.In order to segment images with intensity inhomogeneity efficiently, local intensity information was incorporated into the active contours models 9-11 .For example, Li et al. 9 proposed the regionscalable fitting model which draws upon local intensity means.This model is able to segment object boundaries accurately.In papers 12, 13 , both local intensity means and variances are used to characterize the local intensity distribution in their proposed active contour models.However, these local intensity means and variances must be defined empirically.In addition, similar forms of local intensity means and variances were also introduced in 14 for a statistical interpretation of Mumford-Shah functional 15 .In a word, these methods have a certain capability of handling intensity inhomogeneity.Moreover, there are some models that make full use of advantages of the edge-based and the region-based models, such as the model 7 that unifies the GAC model and the CV model.This model can obtain good results when the contrast between meaningful objects and the background is low.
When the aim is to detect all the objects, the above models are very efficient.However, if only the objects of interest OOI are concerned, the segmentation of OOI using active contour methods becomes difficult.In order to detect OOI, the properties of OOI must be known mathematically.However, it is not an easy task to find out a suitable mathematical description of these characters.Many segmentation methods have used shape or texture characters of OOI.For example, the active shape models ASM 16 , the geodesic active contour method incorporated with shape information 17 , and the supervised texture segmentation method 18 .In this paper, inspired by the weighted bounded variation model 19 , the geodesic active contour model, the region-scalable fitting model and the active contour models based on the Mumford-Shah model 7 , we propose a new model which can be applied to segmenting objects of interest from color images.In order to segment desired objects in color images, the energy functional in our model includes a discrimination function 20 which determines whether an image pixel belongs to the desired objects or not.Moreover, our new model uses not only the edge detector which contains information concerning boundaries of desired objects but also the spatially varying fitting functions which are used to approximate the image intensities.Compared with other active contour models, our method cannot only have the advantages of both edge-based and region-based models, but also avoid the usual drawback in the level set approach e.g., initiation and reinitiation .In particular, our new model can detect the desired objects accurately.Finally, we investigate this new model mathematically: we establish the existence of the minimum to the energy functional, analyze the property of it and implement the numerical algorithm efficiently.
The remainder of the paper is organized as follows: in Section 2, we show some background.In Section 3, our new model is proposed.Theoretical results, iterating schemes and experimental results are also given in this section.Finally, we conclude our paper in Section 4.

The Region-Scalable Fitting Model
Let Ω ⊂ R 2 be the image domain and f : Ω → R be a given gray-scale image.The regionscalable fitting model is defined by minimizing the following energy functional:

2.2
The steady state solution of this gradient flow is

2.3
And it is known that 2.3 is the gradient descent flow of the following energy: 2.4

Some Related Models
In 2 , the geodesic active contour model GAC is defined by the following minimization problem: where L c is the length of the curve c and the function g is an edge indicator function that vanishes at object boundaries.For a color image f f 1 , f 2 , f 3 , a stopping function g x was proposed in 21 where ∧ is the largest eigenvalue of the structure tensor metric g ij in the spatial-spectral space, and where x x 1 , x 2 and R, G, B represent the pixel values of Red, Green, and Blue after Gaussian convolution, respectively, that is, In order to segment the desired objects in color images, a novel stopping function depending on both the discrimination function of OOI and the image gradient was proposed in 20 : where G σ 1 is a Gaussian kernel and the discrimination function α x can be defined by the following steps in 20 .
1 m sample pixels are chosen from the OOI.The color information of the ith sample pixel is denoted by J i1 , J i2 , J i3 .Therefore, J J ij is a m × 3 matrix.
2 Compute the correlation coefficient matrix of J as ρ ρ i,j 3×3 .
5 The confidence interval is constructed as with the degree of confidence 1 − 0 < < 1 , where If a 1 x belongs to the above interval, α x 1.If a 1 x does not belong to the above interval, α x 0.

Our Proposed Model
Let f f 1 , f 2 , f 3 : Ω → R 3 be a given color image where Ω ⊂ R 2 is the image domain a bounded open set .Inspired by the GAC model, the energy functional 2.4 , and the active contour models based on the Mumford-Shah model 7 , our new model is constructed.This model is to minimize the following energy functional w.r.
0 Ω , |ϕ| ≤ g}, g is a diffusion coefficient defined as the formula 2.8 , K σ , G σ 1 are the Gaussian kernels, and α x is a discrimination function.The purpose of constructing such a discrimination function is to derive the characteristics of desired objects so that these characteristics can be shown in the energy functional.This is done by analyzing m sample pixels chosen from the desired objects.The Principal Components Analysis PCA and interval estimation 22 are used in the analysis.By using PCA, a new set of variables which is called principal components, is obtained.By using the interval estimation, we can define an interval that covers many samples for each principal component.In our problem, at most the first two principal components are used.Without loss of generality, we only assume the first principal component is used.An interval a, b for this principal component can be constructed by the interval estimation.When every pixel x of the color image is projected from its RGB values to the first principal component axis, we get a new value u x .If the value is within the interval, the pixel is probabilistically regarded as a pixel in the desired objects.Thus the discrimination function α x based on the color information is constructed as

3.2
In order to understand more details of constructing this discrimination function, please see 23 .
In the following, we analyze the above model from two aspects.Firstly, for any given u, according to the necessary condition of the minimization problem, it is known that the functions f i1 y , f i2 y i 1, 2, 3 must satisfy the following equations: where K σ takes larger values at the points near the center point y, and decreases to 0 as x goes away from y.Therefore, f i1 y , f i2 y i 1, 2, 3 are allowed to vary in space.
Furthermore, for any given f i1 , f i2 i 1, 2, 3 , the model 3.1 can be converted into a simpler form.That is

3.4
If u is limited to a characteristic function 1 Ω c , the energy functional 3.4 can be changed into the following form: where C is a constant.In this case, the model 3.4 is equal to the following constrained minimization problem: when approximating f i of desired objects with spatially varying fitting functions f i1 , f i2 i 1, 2, 3 .
The above analysis shows that our new model uses both the edge detector which contains information concerning boundaries of desired objects, and the spatially varying fitting functions f i1 y , f i2 y i 1, 2, 3 which are used to approximate the image intensities of desired objects.Just because of this, our model can segment the desired objects very accurately.

Mathematical Results
In 7, 24 , the proof of the existence of models has not been given.In the following, we will state the existence of the minimizer to the energy functional 3.4 and analyze the property of it.Theorem 3.1.For any given f i1 , f i2 ∈ L ∞ Ω (i 1, 2, 3), there exists a function u ∈ g −BV 0,1 Ω minimizing the energy functional E1 in 3.4 .

3.7
By 3.9 The structure tensor metric g ij is symmetric positive and ∧ is the largest eigenvalue of g ij .Thus , the BVnorm of u n is bounded.Thus, there is a subsequence, also denoted by {u n }, and u * ∈ BV Ω such that u n → u * strongly in L 1 Ω .
Moreover, according to the formula u n → u * in L 1 Ω , we know that there is a subsequence, also denoted by {u n }, satisfying lim n → ∞ u n x u * x almost everywhere for x ∈ Ω.Since u n x ∈ 0, 1 for any x ∈ Ω, u * x ∈ 0, 1 almost everywhere for x ∈ Ω. Assume

Mathematical Problems in Engineering
Then That is, u n → u * strongly in L 1 Ω and u * x ∈ 0, 1 for every x ∈ Ω.Furthermore, by the lower semicontinuity for the g − BV space, we get

3.15
The formulas 3.14 -3.15 imply the weak lower semicontinuity of the energy functional Therefore, the infimum is attained by u * ∈ g − BV 0,1 Ω and it is a minimum of the energy functional E1.
Similar to 7, 25 , we can obtain a property of minimizers to the energy functional 3.4 .

Theorem 3.2. Let f i x i
1, 2, 3 , g x ∈ 0, 1 .For any given f i1 , f i2 (i 1, 2, 3), if u x is any minimum of the energy functional E1 defined in 3.4 , then for almost every μ ∈ 0, 1 , the characteristic function 1 Ω c 1 {x:u x >μ} is also a global minimum of the functional E1 where c is the boundary of the set Ω c .
In addition, according to the above theorem, we know Ω c {x : u x > μ} is a minimizer of the following minimization problem 3.17

Numerical Implementation
In the numerical algorithm, we do not deal with the new variational model 3.1 directly since too many equations need to be computed.In order to improve computational efficiency, we use the algorithm framework of the paper 24 to deal with our model.That is, we minimize the energy functional E by alternating the following steps.
1 Considering u fixed, compute f i1 and f i2 i 1, 2, 3 by using the formula 3.3 .
2 Considering f i1 and f i2 i 1, 2, 3 fixed, update u by using the iterative schemes of the minimization problem 3.4 .When a steady state is found, the final segmentation is obtained by thresholding u at any level in 0, 1 in our experiments, we choose μ 0.5 .
In the following, we will give the iterative schemes of the minimization problem 3.4 .To solve this minimization problem, we firstly change it into the following unconstrained minimization problem: where ν u max{0, 2|u − 0.5| − 1} is an exact penalty function provided that the constant ζ is chosen large enough.According to the paper 7 , it is known that this unconstrained minimization problem has the same set of minimizers as the minimization problem 3.4 .The energy functional E2 is convex, so it doesn't possess local minimizers.Hence, any minimizer of E2 is global.
Based on 26-28 , a convex regularization is used where θ is chosen to be small enough so that we almost have u v. Thus, we consider the iterative schemes of the above weak approximation as the iterative schemes of step 2 .Since this functional is convex, its minimum can be computed by minimizing this functional with respect to u and v separately.That is 2 u being fixed, we search for v as a solution of

3.21
According to 29 , we know the solution of 3.20 can be given by u v − θ div p, 3.22 where p p 1 , p 2 is given by The previous equation can be solved by the fixed point method p n 1 p n δt∇ div p n − v/θ 1 δt/g x ∇ div p n − v/θ , p 0 0, 0 .

3.26
In this paper, our model is solved numerically by alternative minimization schemes and not by using the classical Euler-Lagrange equations method.When using Euler-Lagrange equations, one must consider Ω g x |∇u| 2 ε 2 dx instead of the weighted TV-norm, where the small parameter ε is necessary to prevent numerical instabilities.The direct result of this regularization parameter is the obligation to use a small temporal step to ensure a correct minimization process.Thus, a large number of iterations to reach the steady state solution is necessary.In other words, although it is correct, the segmentation process remains slow.In the algorithm of our paper, the iteration process 3.22 -3.24 is very similar to the standard Chambolle's projection algorithm.The only difference is the appearance of the factor g x .This iteration process has two important advantages: firstly, it can be implemented very fast since it uses many useful convex optimization tools.Secondly, it is more faithful to the continuous formulation of the energy since it does not use the additional artificial parameter ε.Therefore, our algorithm is fast and doesn't need so many iterations to reach the steady state.Moreover, the algorithm of our paper is numerically more efficient than classical level set methods since the level set methods need to initialize the active contour in a distance function and reinitialize it periodically during the evolution.In fact, the algorithm of our paper belongs to the algorithm framework mentioned in 24 .And there are many papers 7, 28 that have analyzed the complexity of this kind of algorithms.So we just analyze it simply in this paper.If more details are wanted, please see 7, 24, 28 .

Experimental Results
Our method can be implemented on many synthetic and natural images.First, we consider two different cases Figures 1 a and 2 a .Figure 1 a is an image containing three different objects.The blue rectangle is our desired object.Figure 2 a is not a simple image where the bird is our interested object.Figures 1 b , 2 b , 1 c , and 2 c display the final active contours and u got by using our new model, respectively.From these experimental results, we find our model can detect OOI, regardless of other objects.
In the following, we compare our new model with the model 23 in Figures 3, 4, and 5. Figures 3 a , 4 a , and 5 a are the original images.Our purpose is to segment the plane in Figure 3 a , the yellow leaves in Figure 4 a and the green leaves of oranges in Figure 5 a .Figures 3 b , 4 b , 5 b , 3 c , 4 c , and 5 c denote the final active contours and u obtained by using our new model, respectively.According to these results, we see that our model can detect the boundaries of the desired objects very accurately.And these results are hard to achieve by using the model in 23 .

Conclusion
This paper describes a new variational model for segmenting desired objects in color images.This model is inspired by the GAC model, the region-scalable fitting model, the weighted bounded variation model and the active contour models based on the Mumford-Shah model.In order to segment objects of interest from color images, the energy functional in our model includes a discrimination function which determines whether an image pixel belongs to the desired objects or not.Compared with other active contour models, our new model cannot only avoid the usual drawback in the level set approach but also detect the desired objects accurately.Our numerical results confirm the effectiveness of our algorithm.Moreover, we establish the existence of the minimum to the new energy functional and analyze the property of it.

Figure 3 :
Figure 3: An example of an image segmentation.a The original image our desired object is the plane .b-c The final active contour and u obtained by using our model m 7, δt 1/8, λ i1 λ i2 0.001 i 1, 2, 3 , θ 0.1 .d The final active contour obtained by using the model in 23 .

Figure 4 :Figure 5 :
Figure 4: An example of an image segmentation.a The original image our desired objects are the yellow leaves .b-c The final active contour and u obtained by using our model m 7, δt 1/8, λ i1 λ i2 0.001 i 1, 2, 3 , θ 0.1 .d The final active contour obtained by using the model in 23 .