Robust Evolution Method of Active Contour Models and Application in Segmentation of Image Sequence

. Active contour models are widely used in image segmentation. In order to obtain ideal object boundary, researchers utilize various information to define new models for image segmentation. However, the models could not meet all scenes of image. In this paper, we propose a block evolution method to improve the robustness of contour evolution. A block matrix is consisted of contours of former iterations and contours of shape prior, and a nuclear norm of the matrix is a measure of the similarity of these shapes. The constraint of the nuclear norm minimization is imposed on the evolution of active contour models, which could avoid large deformation of the adjacent curves and keep the shape conformability of contour in the evolution. The shape prior can be integrated into the block evolution method, which is effective in dealing with missing features of images and noise. The proposed method can be applied to image sequence segmentation. Experiments demonstrate that the proposed method improves the robust performance of active contour models and can increase the flexibility of applications in image sequence segmentation.


Introduction
Object extraction and image segmentation [1] are an important and fundamental topics in computer vision and image processing.Snakes or active contour models (ACM) [2] which have shown their great performances are the key methods for image segmentation.The principal idea in snakes is to obtain an optimum by minimizing an energy functional.A contour is evolved by minimizing some certain energies to match the object boundary while preserving the smoothness of the contour.There are broadly two types of active contour models according to the representation of the curve, that is, parametric active contours [2] and implicit active contours [3][4][5].The active contour is usually represented by landmarks in parametric active contours and an energy functional was originally introduced by Kass et al. [2], while contours in implicit approaches are represented by level set [3,5], which offers great flexibility for the curve topology.The numerical computations of evolving level set function can be elegantly performed by using the mature numerical algorithm of partial differential equations (PDE) [6].
Contours always evolve to major deviations from true object boundary.One of the reasons is that various models could not meet all scenes, such as noise, inhomogeneous intensity [7,8], and missing features in images; another one of the reasons is the mode of deformation of contours, and contours may suffer from undesired location and could not escape the local minimization [9,10] because of improper mode of deformation.In implicit active contour model, contour represented by level sets may extract unnecessary objects.Though the energy functionals of some active contour models are convex [11,12], some unnecessary components (such as noises) are also extracted.For parametric active contour model, contours can be not smooth or appear as self-crossing because of noise or improper parameters.As shown in Figure 1, the image is polluted by noise.The deformation of contour is shown and evolving contour appear as self-crossing within few iterations, which can cause the contour converge to undesired results.
To improve the robustness of active contours, the shape prior [13][14][15][16][17] is often used.The prior knowledge of the shape to be segmented is modeled based on a set of manually annotated shapes to guide the segmentation.In recent works, the shape prior was applied by regularizing the distance from the active contour to the template in a level set framework.The shape prior has proven to be a powerful tool in segmentation.However, the shape prior is learnt from a set of annotated data.As pointed by [18], existing shapes in the training set are difficult to model the new instance in the testing images.Recently, the active contour with group similarity (ACGS) is proposed by Zhou et al. [18].In ACGS, the shape prior is not learnt, the shapes of evolving contour in several frame images are utilized to constrain the evolution of contour.ACGS is a better method to evolve the contour, which is effective in dealing with missing features in images.
In this paper, we propose a robust evolution method to keep the similarity among evolving sequence contours for parametric active contours model.The shape consistence of the deforming contour is kept in the evolution.Large deformation of the adjacent curves is avoided.In summary, the contributions of this paper are as follows: (1) A block evolution mode is proposed, where a block is consisted by contours of former iterations or contours of shape prior.
(2) The proposed algorithm has generality, which could be conveniently integrated into active contour models, including the shape prior based active contour model.
(3) The proposed method also has flexibility, which can be used to image sequence segmentation.
The rest of this paper is organized as follows: Section 2 introduces the model and algorithm of our method.The proposed algorithm is extended to the active contour model with shape prior in Section 3. Section 4 demonstrates the merits of our method by experiments.Finally, Section 5 concludes the paper with some discussions.

Proposed Method
In active contour model, an energy functional is usually minimized to segment the objects.For the energy functional () with respect to , the curve  is viewed as the object when the energy functional (C) obtains its minimum.() is minimized as follows: min   () . ( The deformation or evolution equation of curve  is obtained with calculus of variations and gradient descent method, that is, where ∇ is the gradient of function ,  is the time variable, and  is a parameter.The discrete form of the above equation is as follows: where   represents the th iterative solution and Δ is the time step.The converged curve   is viewed as the object boundary through a sequence of evolution  1 , . . .,   .From (3),  +1 is determined by   and the current gradient vector ∇(  ).Because of influences of noises, parameters and nonconvexity of energy functional, the gradient descent direction of ∇(  ) may have derivation from ideal gradient direction, which causes  +1 to suffer from a local minimum.Various of methods are proposed to define novel energy functional [19][20][21][22][23] or introduce some optimal methods [24][25][26][27] for obtaining robust ideal results.However, to our knowledge, few researchers try to change or improve the evolution mode of contours.

Energy Functional Model.
Based on the above analysis, a block evolution method of curve is proposed by imposing a constraint into active contour model.To keep the shape conformability of deforming contours, the following equation is considered: Corresponding to the curve , the set  = [ 1 , . . .,   ] is a block constituted by several curves, and  is the cardinality of . is the energy functional of an active contour model to evolve the contour, and () = ∑  =1 (  ).Through a sequence of evolution  1 , . . .,   , the converged contour   is viewed as the final result.
The goal is to robustly evolve the curve converging to object.Thus, an energy functional keeping the consistence of evolving contours is integrated into .A nuclear norm ‖‖ * , that is, the sum of singular values of  is considered.The nuclear norm is a continuous function and convex; some fast algorithms could be utilized.The energy functional of the proposed model is defined as follows: Since  is a block including several contours and contour  is represented by parametric curve in the 2D plane,  = [ 1 , . . .,   ,  1 , . . .,   ]  ∈  2 ,  is the number of landmarks, and (  ,   ) is a landmark on the curve.Thus, the size of  is 2 × .From the above equation, because the constraint of ‖‖ * is imposed, the shape conformability is kept and the robustness of evolution is improved. is the energy functional of a general active contour model, such as the parametric C-V model [23].The parametric C-V model is defined as follows: where  is the gray intensity, Ω 1 and Ω 2 are the regions inside and outside the contour , where   represents the th iterative solution and Δ is the time step.The converged contour   is viewed as the final result through a sequence of evolution  1 , . . .,   .From the above equation, the current  +1 is determined by   and the gradient ∇  .Since  is block of several contours, the evolution mode is called block evolution method.In a block, the contours of former iterations are utilized to avoid large deformation of evolving contours.Compared with ACGS, temporal sequence relationship in  1 , . . .,   for dealing with one image with the proposed model existed, while ACGS uses multiple images to evolve contours, respectively, and there is no temporal sequence in evolving contours with ACGS.Thus, the proposed method aims to keep the deformation robustness of contour.Furthermore, the prior information can be flexibly utilized to guide the evolution with the proposed method.

Algorithm.
In order to solve (5), a regularized method is considered. is a differentiable function and ‖‖ * is a convex function.For the linear combination of  and ‖‖ * , Proximal Gradient (PG) method is always used to solve the problem.The PG uses quadratic approximation to () based on the previous estimate   at each iteration.
where ‖ ⋅ ‖  denotes the Frobenius norm,  is a constant, and  = .By introducing a variable , the above equation becomes where  is updated by   =   − (1/)∇(  ).Therefore, the key problem is to solve (9).For (9), it has been proved that it can be solved by the following expression: where   () is the singular value thresholding operator; it is given as follows: where   and V  are the left and right singular vectors of ,   is the singular value, and (⋅) + = max (⋅, 0).
For Algorithm 1, it is not necessary to have explicit representation for the energy function ; the gradient ∇ of energy functional  is utilized from Algorithm 2. Therefore, block evolution method can be integrated into the active Input: initial contour  0 , ; for  = 0 to maximum number of iterations do for  = 1 to  do Algorithm 1: Block evolution algorithm.
Input: initial contour  0 , , and shape prior   ; for  = 0 to maximum number of iterations do for  = 1 to  do contour model based on external force field  [28][29][30][31][32], such as gradient vector field (GVF) [28], vector field convolution (VFC) [30] external forces, and just replacing ∇ with external force field  according to (8).The proposed algorithm has advantage of evolution robustness, since it utilizes the information of previous several iterations.

Extension
In the above section, the block evolution method is integrated into active contour model.For the active contour model with shape prior, the block evolution method is also expanded to the active contour model with shape prior.For a shape prior   , the energy functional of active contour model with shape prior   is usually defined as follows: where  is the linear or affine transform, such as translation, scaling, and rotation.It is necessary to compute the parameters of  at each iteration.The evolution with shape prior is computed by minimizing the above energy functional: The shape prior gradient   −   (  ) is integrated to energy gradient descent direction of ∇(  ) to constraint the shape of deforming contour.On the other hand, the above equation can be written as follows: The active contour model with shape prior has better performance because of dynamically utilizing the information of   and   .Thus, the block evolution method is also integrated into active contour model with shape prior   by dynamically using the contour information   and   .Similar to Algorithm 1, the block evolution algorithm for active contour model with shape prior is given as shown in Algorithm 2.
In fact, it is easy to extend to the active contour model with prior shapes of multiple contours.Corresponding to the program statement   = (  ,   ) of Algorithm 2, the extended algorithm for active contour model with prior shapes is obtained by setting statement   = (  ,   1 , . . .,    ) of Algorithm 2, where   1 , . . .,    are the prior shapes.In order to keep the consistence, we still use   representing   = (  1 , . . .,    ) and Algorithm 1 is extended to the active contour model with prior shape of multiple contours.
According to the iteration   = (  ,   ) and  +1 =   (  ) in proposed algorithm, the  +1 is determined by (  ,   ) and not only determined by   .In the block of (  ,   ), the   in block of (  ,   ) is the shape prior, which has advantages in dealing with missing features of objects and some complex background of image, such as noise.For sequence image segmentation, the segmented objects for several frames can be viewed as the shape prior.Therefore, the extended Algorithm 2 for active contour model with shape prior can be applied to segmentation for sequence image [18,33] and object tracking [34][35][36][37].
An example is shown to verify the effectiveness of the proposed method, which is shown in Figure 2. Setting the same parameters in Figure 1, active contour model with the block evolution method is used to segment the object.As shown in Figure 2, the parameter  = 3 is set.The object boundary is extracted with only 3 block iterations.The block evolution method without shape prior is tested, and the nuclear norm is utilized to constraint the shape conformability of evolving contours.The evolution is robust and the object boundary is converged.Compared with the evolution utilizing parametric C-V model, the block evolution method is more robust in the evolution.

Experiments and Analysis
To demonstrate the advantages of the block evolution method, we compare the results of the same active contour model before and after applying the proposed constraint.Then, the proposed method is compared with ACGS.If there is no other statement, the region based active contour, that is, the parametric C-V model, is selected as the basic model.

Experimental Results.
In this section, several images are synthesized to test the performances of proposed method.A heart shape is the object.There are occlusions or deletions to be added in these images.The task is to extract the heart shapes in all tested images.Our proposed algorithm and the parametric C-V model are tested.As shown in Figure 3, the result is obtained with the parametric C-V model.The block evolution method with shape prior is shown in Figure 5, and the block evolution method with 6 shape priors is shown in Figure 4. From the comparisons, the block evolution method with multiple shape prior is robust against the occlusion and deletion.In contrast, the parametric C-V model is sensitive to occlusions and missing features in segmenting images.As shown in Figure 3, the shapes of extracted boundaries with the parametric C-V model are different.In the proposed method, the shape constraint is imposed and the shape consistence of evolving contours is kept.This is the reason why the proposed method is robust to occlusions or missing features of images.
The proposed algorithm is especially fit to apply to segmentation of image sequences since the extracted object boundary in the frame is viewed as the prior information in segmenting the next frame.Some image sequences are utilized to test the ability of the proposed method.Two typical image sequences show the performances of the proposed method; there are continuous frames in one image sequence; and there are discontinuous image frames in the other image sequence.
In the first image sequence, the continuous ten image frames are selected, which are shown in Figure 6, and the size of each image in this sequence is 352 × 240.The results with the proposed method and the parametric C-V model are shown in Figures 8 and 7, respectively.The extracted object boundaries with both methods are compared in Figure 9.The fish in these images is blurry, which is difficult to extract.From the results in Figures 8 and 7, respectively, both of these tested methods roughly converge to the contour of the object.However, there are still some differences between the extracted results.As shown in Figure 9, (a) are the results with the parametric C-V model and (b) are the results with the proposed method.The converged contours are nonsmooth, appear as double-contours and self-crossing, which is clearly shown in Figure 10(a).While the proposed method utilizes the shape information to constrain the evolution of contours, the extracted contour is viewed as the shape prior in the next image frame.Because of the constraint of shapes, the shapes of evolving contour are always smooth in the evolution, which is shown in the second row of Figure 10.
Because of the nuclear norm to constrain the evolving contours, the shapes of the deforming contour keep consistence throughout evolution of contours.Therefore, the converged contour is always smooth, as shown in the second row of Figure 10.On the other hand, since utilizing the shape      information and nuclear norm, the proposed algorithm is robust and the accuracy of extracted results with proposed method is more precious compared with the typical active contour model.According to Figure 10, the enlarged results are shown; it seems that the proposed method obtains better object boundaries compared with the tested method.In order to evaluate the performances and accuracies of tested methods effectively, the quantitative results are given.For each tested algorithm, the traditional -measure score [38] is the weighted, harmonic mean of precision and recall values; it is usually used to evaluate the quantitative results.It is given as follows: The average -measure score (mean ± standard deviation) for each algorithm is the following: the tested method: 0.82 ± 0.08 and proposed method: 0.93 ± 0.05.According to  score, the proposed method outperforms the parametric C-V model in extracting object boundary.

Comparison with ACGS in Sequence Image Segmentation.
In this part, the proposed algorithm and ACGS are tested and compared.As analyzed in the Section 3, because of integrating prior information, the proposed method has better flexibility in segmentation of image sequence.Eight images are continuously selected to form an image sequence to test algorithms, as shown in Figure 11.The object in these images is a ball and the locations of ball in each image frame are different; the shapes of objects in these images are   the same.ACGS offers an effective way to image sequence segmentation.In ACGS, the contours must be simultaneously initialized in every tested sequence images.Thus, the location information of objects in the current images is useless in the following images.Once the initialization of contours is undesired, the results with ACGS is always unsatisfactory.As shown in the Figure 12(a), ACGS failed to extract the objects in some images.A typical example is the first image of second row in Figure 12(a), and the contour is completely outside the object.As shown in Figure 12(b), ACGS with careful initialization obtains better results compared with Figure 12(a).By contrast, the proposed method obtains better results, which is shown in Figure 12(c).The proposed method allows the final result in current frame acting initial contour in the next frame.Therefore, the proposed method always obtains better accuracy of extracting objects compared with ACGS.

Conclusion
In this paper, a method of active contour with block evolution method is proposed.In the proposed method, the shape prior can be integrated, and the shapes similarity of evolving contours are kept.On the other hand, evolving contour deforms robustly with the proposed method because of utilizing the contour's evolution result of the previous iteration steps.Simulations show that the proposed method has the robust performances in segmenting objects.Compared with the state-of-the-art method (ACGS), the proposed method has shown flexibility of application in segmentation of image sequence.

Figure 1 :
Figure 1: Contour evolves within few iterations; self-crossing of contour appears, which could make the contour suffer from undesired location.

Figure 3 :
Figure 3: The results with the parametric C-V model.

Figure 5 :
Figure 5: The results with the proposed method.

Figure 7 :
Figure 7: The results of segmentation with the parametric C-V model.

Figure 8 :
Figure 8: The results of segmentation with proposed block evolution method.

Figure 9 :Figure 10 :
Figure 9: The compared results of segmentation.(a) are the results with the parametric C-V model, and (b) are the results with proposed method.

Figure 12 :
Figure 12: The results of object tracking (including eight images) with ACGS and the proposed method.(a) is the result with ACGS, (b) is the result with ACGS through careful initialization, and (c) is the result with the proposed method.
1and  2 are the mean intensities of Ω 1 and Ω 2 , respectively, length() represents the length of contour , and  is a parameter. cv is usually less sensitive to initialization and has fewer parameters to tune. makes the contour evolving to object boundary.‖‖ * is a tight convex surrogate to the rank operator, which keeps the elements of  similar.With  and nuclear norm ‖‖ * , contour evolves robustly into the object boundary.Similar to (2), the evolution equation of  is computed as follows: