This paper presents a variation of the fuzzy local information c-means clustering (FLICM) algorithm that provides color texture image clustering. The proposed algorithm incorporates region-level spatial, spectral, and structural information in a novel fuzzy way. The new algorithm, called RFLICM, combines FLICM and region-level Markov random field model (RMRF) together to make use of large scale interactions between image patches instead of pixels. RFLICM can overcome the weakness of FLICM when dealing with textured images and at the same time enhances the clustering performance. The major characteristic of RFLICM is the use of a region-level fuzzy factor, aiming to guarantee texture homogeneity and preserve region boundaries. Experiments performed on synthetic and remote sensing images show that RFLICM is effective in providing accuracy to color texture images.
1. Introduction
Image segmentation is one of the most important tasks in computer vision, and many other fields of application are closely related to it, including pattern recognition, remote sensing, and medical diagnostics. The purpose of segmentation is to partition an image into homogeneous regions. Although many methods have been proposed to solve this problem, the fuzzy c-means algorithm (FCM) [1] has been widely applied to image segmentation, because its fuzzy nature allows more original information being considered. FCM has some obvious advantages such as the straightforward implementation, the fairly robust behavior, the applicability to multichannel data, and the ability to model uncertainty within the data [2]. However, the traditional FCM failed to consider the spatial information, which may cause poor results when dealing with images corrupted by noise, outliers, and other image artifacts.
To compensate this drawback of FCM, many attempts have been proposed, including adding a preprocessing image smoothing step before clustering [3] or incorporating spatial context in different ways, for example, the fuzzy membership function refinement [4, 5], the dissimilarity function improvement [6], the objective function regularization [2, 7], and the fuzzy treatment of hidden Markov random field (MRF) model-based image segmentation (HMRF-FCM) [8]. More recently, following the method of objective function regularization, Krinidis and Chatzis [9] presented a robust image clustering method called fuzzy local information c-means (FLICM). In FLICM, the clustering is dependent on both the spectral and local spatial information which cooperate by using a fuzzy factor. However, this algorithm assumes that the label of one pixel is only related to the labels of its neighboring pixels. Therefore, only interactions between neighboring pixels can be used, which makes the algorithm defective in dealing with color texture images due to the lack of information of large scale interactions between image patches instead of pixels.
In recent years, there is an increasing trend to analyze images based on image regions in order to make use of different kinds of large scale local information (e.g., spectral, spatial, and structural information). The MRF model is one of the most popular methods to integrate these kinds of information together, because of its powerful capability of describing the continuity of image characteristics. Based on region-level MRF (RMRF), Yang and Jiang [10, 11] proposed a gray image segmentation method and showed good performance; Clausi et al.’s research group also presented algorithms to deal with gray images [12], multivariate images [13], and polarimetric SAR images [14]. All of these methods have shown the superiority to pixel-based ones.
Inspired by the success of RMRF-based image segmentation methods in image segmentation, this paper improves FLICM into a region-based version, named RFLICM. In RFLICM, a novel fuzzy factor is defined by the basic idea of RMRF and incorporated into the objective function of FCM. The fuzzy factor can simultaneously incorporate the region-level spatial, structural, and spectral information in a fuzzy way and helps to guarantee the texture homogeneity as well as preserving region boundaries. All these characteristics make RFLICM more general and suitable for color texture image segmentation.
The remainder of the paper is organized as follows. Section 2 briefly describes the fuzzy local information c-means algorithm, followed by the basic theory of RMRF-based image segmentation. The RFLICM algorithm is introduced in Section 3. Experimental results are presented in Section 4 and conclusions are drawn in Section 5.
2. Related Work2.1. Fuzzy Local Information c-Means (FLICM) Clustering Algorithm
In [9], a fuzzy factor is used to incorporate local spatial and gray level information into the objective function of FCM:(1)Jm=∑i=1N∑k=1cuikmd2xi,vk+Gki,where X={xi∣1≤i≤N,xi⊆RB} denotes an N-pixel image defined on a rectangular lattice set in the B-dimensional vector space, N is the number of image pixels, c is the number of clusters with 2≤c≤N, uik∈[01] is the degree of membership of xi in the kth cluster, m is the weighting exponent on each fuzzy membership, vk is the prototype of the center of cluster k, d2(xi,vk) is a distance measure between object xi and cluster center vk, and Gki is the fuzzy factor which is defined as(2)Gki=∑j∈Nij≠i1dij+11-ujkmxj-vk2,where ith pixel is the center of the local window (e.g., 3 × 3), k is the reference cluster, and pixel j belongs to the set of neighboring pixels falling into the window around the ith pixel (denoted as Ni). dij is the spatial Euclidean distance between pixel i and j and ujk is the membership of the jth pixel in the kth cluster.
Local minimum extreme of Jm is obtained iteratively as follows.
Step 1.
Set clusters number c, fuzzification parameter m, and the stopping condition ϵ.
Step 2.
Initialize randomly the fuzzy partition matrix.
Step 3.
Set the loop counter b=0.
Step 4.
Calculate the cluster prototype using(3)vk=∑i=1Nukimxi∑i=1Nukim.
If max{U(b)-U(b+1)}<ϵ, then stop; otherwise, set b=b+1 and go to Step 4.
2.2. RMRF-Based Image Segmentation
Assume the image has been over-segmented previously into NR disjoint regions. In region-level methods [11–14], all pixels in the same region are assumed to have the same label. Let C=Cr1,Cr2,…,CrNR be the label image. In the Bayesian image segmentation framework, the segmented image C is:(5)C=argmaxCP(X∣C)P(C),where P(X∣C) and P(C) are the class conditional probability and prior probability, respectively. With the assumption that the noise in the image is independent Gaussian white noise, P(X∣C) can be written as follows:(6)P(X∣C)=∏i=1Q∏ij∈rigxij;μCri,ΣCri,where g(xij;μCri,ΣCri) is the Gaussian probability density function with mean value μCri and covariance matrix ΣCri. And if RMRF is used to model the prior probability, P(C) can be described as:(7)PC=1Zexp-UCT,where Z=∑Cexp{-U(C)/T} is a normalizing constant and U(C) is an energy function defined as:(8)UC=∑i=1NR∑rj∈ηriVCri,Crj,where ηri is the set of neighboring regions of ri, and V(Cri,Crj) is defined as [10]:(9)VCri,Crj=bijaiδCri,Crjbiμi-μj,where bij is the length of the common boundaries between region ri and rj, bi=∑rj∈ηibij is the boundary length of region ri, ai is the area of region ri, μi and μj are the mean value of region ri and rj, respectively, and δ·,· is the Kronecker function.
Then the configuration of C can be determined by(10)C=argminC∑i=1Q-∑ij∈riloggxij;μCri,ΣCri∑rj∈ηibijaiδCri,Crjbiμi-μj+β∑rj∈ηibijaiδCri,Crjbiμi-μj,where β is the potential function. Equation (10) can be solved by simulated annealing (SA) [16] or iterative conditional mode (ICM) [17]. For the sake of efficiency, ICM is a good choice. In this paper, we denote the ICM algorithm based on the RMRF as RICM. By using a RMRF model-based prior probability, RICM can make use of the large scale local spectral, spatial, and structural information during the segmentation process.
3. Region-Level Fuzzy Local Information c-Means (FLICM) Clustering Algorithm
Motivated by the successful application of RMRF to image segmentation, we propose, in this paper, a novel framework for image clustering by extending FLICM to a region-based version, called RFLICM.
For each region, some region-level features are extracted firstly. For example, μi and Σi are the mean value and covariance matrix of ri, respectively, ai and bi are the area and boundary length of ri, respectively, and bij is the common boundary length between ri and rj, satisfying the constraint of bi=∑j∈ηibij.
3.1. The Region-Level Fuzzy Factor
In order to overcome the defect of FLICM on using large scale information, the new fuzzy factor should take into account the region-level spatial, spectral, and structural information. Following the basic idea of RMRF, the fuzzy factor for each region is only determined by its direct neighboring region set ηi. So, in this paper, the novel region-level fuzzy factor for region ri is defined as(11)RGkri=∑j∈ηibijaibi+bjiajbjurik-urjkmμi-μj,where urjk is the membership value of rj belonging to kth cluster.
It is easy to see that the factor RGkri makes the influence of regions within the direct neighborhood ηi, to change flexibly according to their areas, boundary lengths, and mean values. Therefore, more region-level information can be used to deal with color texture segmentation. In (11), μi-μj acts like a normalizing coefficient preventing bijai/bi+bjiaj/bj to have a too large value. Similar to Gkri, (2), RGkri also reflects the damping extent of the neighboring regions to be assigned to different labels. So the designed fuzzy factor has the ability to force neighboring regions to be equally labeled.
3.2. General Framework of RFLICM
We assume that all pixels in the same region have the same membership value belonging to different clusters. So the objective function RFLICM is defined as(12)Jm=∑i=1RN∑k=1curikmd2ri,vk+β×RGkri,where β is a constant weighting parameter, it has the similar function as the potential function in RMRF-based image segmentation, (10). The first term in the right part of (12) depicts the fidelity of clustering result to image data, and the second term is a smooth term that forces neighboring regions to have the same label. The parameter β controls the contribution of these two terms.
In this paper, we assume that pixels in each cluster follow a Gaussian distribution. So the distance measure d(ri,vk) is defined as(13)d2ri,vk=-12B∑ij∈rilog(g(xij;μk,Σk)),where vk and Σk are the mean value and covariance matrix of the kth cluster, respectively, and B is the spectral number of the image being processed.
Similar to FLICM, a solution of the objective function of RFLICM, (12), can be obtained through an iterative process, which is carried as follows.
Step 1.
Get the initial partition of the input image.
Step 2.
Set the number c of the cluster prototypes, fuzzification parameter m, weighting parameter β, and the stopping condition ϵ.
Step 3.
Set the loop number b=0, perform FCM clustering on image pixels, and get the initial region level fuzzy partition matrix U(0) based on the minimizing membership the rule:(14)urik(0)=minij∈riuijk,where uijk is the membership value of pixel ij belonging to the kth cluster.
Step 4.
Calculate the cluster prototypes using(15)vk=∑i=1RNurikmaiμi∑i=1RNaiurikm,Σk=∑i=1RNurikm∑ij∈rixij-vkxij-vkT∑i=1RNaiurikm.
Step 5.
Calculate the membership functions using(16)urik=1∑j=1cd2ri,vk+Gkri/d2ri,vj+Gjri(1/(m-1)).
Step 6.
If max{U(b)-U(b+1)}<ϵ, then stop; otherwise, set b=b+1 and go to Step 4.
When the algorithm has converged, a defuzzification process of maximum membership is employed to convert the partition matrix U to a segmentation result:(17)Cri=argmaxk{urik},k=1,2,…,c.
A constraint is imposed on the membership value urik in (15) to avoid the underfitting of Gaussian parameters:(18)urik=urik,urik≥1c,0,otherwise.
Another issue that is worthy to be pointed out is the determination of the weighting parameter β. It controls the smooth strength of the region-level fuzzy factor. In this paper, we employ an increasing schedule for β to guarantee the accuracy of parameter estimation and the homogeneity of segmentation result. It increases with the iterative procedure:(19)β(b)=β(0)+0.9blog2b-1.At the first stage of the iteration, β takes a small value to depress the influence of the region-level fuzzy factor Grik, which forces the estimated Gaussian parameter to fit well the image data. As β increases, more large scale local spectral, spatial, and structural information is introduced into the clustering procedure. Finally, β reaches its limitation, the iteration procedure arrives at a balance state, and Grik begins to help to refine the final result.
4. Experimental Results
In this section, we show the performance of the proposed method by presenting results on synthetic images and remote sensing images. Furthermore, we compare the efficiency of RFLICM with HMRF-FCM [8], FLICM [9], and RICM described in Section 2.2. For FLICM, we set the window size as 3 × 3. For HMRF-FCM, we use the 2nd neighborhood. For both RICM and our proposed RFLICM, mean shift algorithm (MS) [18] is chosen to get the initial partition. In this experiment, we set β(0)=10 for our algorithm and set different β for RICM using the trial-and-error method (we select the parameter in the range of [110] with the interval of 0.5, and the one with the highest segmentation accuracy is recorded and is listed in Table 1). For MS, it is difficult to choose the optimal parameter set. How to choose good parameters is out of the scope of this paper. Intuitively, if parameters are set too large, there will be a risk of losing useful information in the initial oversegmentation; otherwise, large scale interactions cannot be obtained and employed properly. In this paper, we just experimentally choose both the spatial resolution parameter and the range resolution parameter to be 6 and the size of the smallest segment to be 10.
Different β selected for RICM.
Image
Figure 1(a)
Synthetic color texture images in Figure 2
Figure 3(a)
Figure 4(a)
The 1st one
The 2nd one
The 3rd one
The 4th one
β
2.5
4.0
10.0
2.0
2.0
1.5
2.0
In order to quantitatively evaluate the proposed method, both accuracy acc and Kappa coefficient kc are used as indicators [19]. If Pij is the proportion of subjects that were assigned to the ith class by the first image and the jth class by the second image, denote Pi·=∑j=1cPij and P·j=∑i=1cPij, the values of acc and kc are defined as(20)acc=∑k=1cPkk,kc=∑k=1cPkk-∑k=1cP·kPk·1-∑k=1cP·kPk·.
First of all, we apply these methods to a synthetic noisy image shown in Figure 1(a), which was obtained by adding Gaussian noise (zero mean noise with 0.06 variance) to a four-color checkerboard image shown in Figure 1(b). The size of the noisy image is 128 × 128, and the gray levels of four desired regions are 0, 0.3333, 0.6667, and 1.0, respectively. The segmentation results are shown in Figure 1. From Figures 1(c) and 1(d), one can find that both FLICM and HMRF-FCM can separate the image into proper regions but still have some obvious misclassifications and seriously smoothed boundaries, while Figures 1(e) and 1(f) show that RICM and RFLICM can obtain more accurate boundaries and less misclassifications. The most interesting comparison can be found between the result of RFLICM (Figure 1(f)) and those of referenced methods (Figures 1(c)–1(d)); it is obvious that RFLICM shows the best performance to resist image noise. The corresponding acc and Kappa are listed in Table 2. The quantitative indicators also show that the proposed method is more robust than the referenced ones.
Comparison accuracies of four different algorithms on the noisy image in Figure 1 (acc/kc).
Method
FLICM
HMRF-FCM
RICM
RFLICM
acc/kc
0.8948/0.8277
0.9511/0.9301
0.9603/0.9584
0.9708/0.9692
Segmentation results of a synthetic noise image. (a) The synthetic noisy image; (b) the ground truth image; (c) the result of FLICM; (d) the result of HMRF-FCM; (e) the result of RICM; and (f) the result of RFLICM.
Secondly, we apply these four algorithms to a group of synthetic color texture image, which are shown in the first row of Figure 2. The first three images are obtained from the Prague Texture Segmentation Data generator [15], and the last one is synthetized from four different kinds of remote sensing textures. The 2nd row shows the corresponding ground truth images. The segmentation results of FLICM, HMRF-FCM, RICM, and RFLICM are shown in the 3rd, the 4th, the 5th, and the last row in Figure 2, respectively.
Results of synthetic color texture images. The 1st row shows four synthetic color images. The first three images are obtained from the Prague Texture Segmentation Data generator [15] and the last one is synthetized from four different kinds of remote sensing textures. The 2nd row shows the corresponding ground truth images. The 3rd, 4th, 5th, and last row show the segmentation results from FLICM, HMRF-FCM, RICM, and the proposed method, respectively.
From the segmentation results, it is easy to be seen that both FLICM and HMRF-FCM cannot get acceptable results: most texture areas are wrongly labeled. The main reason is that they only employ the interactions between image pixels. Therefore, there is not enough information to be used for describing texture models in images. The RICM algorithm can get rather better results than FLICM and HMRF-FCM. But it is very easy to trap into local minima, which results in inconsistent segmentation results, while, for all of these textured images, our proposed method can get rather satisfactory results. As shown in the last row in Figure 2, the segmentation results exhibit homogeneous texture regions and also pretty good boundaries. Such results have verified the basic idea of this paper to improve FLICM into a region-based version. Table 3 also gives the quantitative comparison results of these four algorithms on these synthetic color texture images. The highest values are highlighted in the table. It is clearly illustrated that the proposed RFLICM algorithm gives rise to better texture segmentation performance than the referenced ones.
Comparison accuracies of four different algorithms on the synthetic color texture images in Figure 2 (acc/kc).
The 1st image
The 2nd image
The 3rd image
The 4th image
Average
FLICM
0.4819/0.2282
0.6681/0.4849
0.6501/0.5470
0.4556/0.3036
0.5639/0.3909
HMRF-FCM
0.8941/0.8408
0.8093/0.7136
0.9090/0.8774
0.8732/0.8123
0.8714/0.8110
RICM
0.9998/0.9901
0.9596/0.9387
0.8520/0.7926
0.9506/0.9262
0.9405/0.9119
RFLICM
0.9998/0.9901
0.9894/0.9838
0.9887/0.9849
0.9723/0.9583
0.9876/0.9793
Besides, we also applied the same four algorithms on the multispectral high resolution remote sensing images, which are shown in Figures 3(a) and 4(a), respectively. Figure 3(a) is a piece of 256 × 256 Quickbird image of an outskirt in Wuhan, and Figure 4(a) is a 438 × 438 Spot5 image of Pingshuo area, China. Their segmentation results are shown in Figures 3 and 4, respectively. And the corresponding quantitative measurements are recorded in Table 4.
Comparison accuracies of four different algorithms on the high resolution remote sensing images (acc/kc).
Figure 3(a)
Figure 4(a)
Average
FLICM
0.5129/0.2566
0.6379/0.4909
0.5754/0.3738
HMRF-FCM
0.6340/0.4692
0.7112/0.5904
0.6726/0.5298
RICM
0.8638/0.7856
0.7850/0.6935
0.7936/0.7395
RFLICM
0.8810/0.8121
0.8734/0.8148
0.8772/0.8135
Segmentation results of a QuickBird image. (a) The Quickbird image of an outskirt in Wuhan City, China; (b) the ground truth image; (c) the result of FLICM; (d) the result of HMRF-FCM; (e) the result of RICM; and (f) the result of RFLICM.
Segmentation results of a Spot5 image. (a) The Spot5 image of Pingshuo area, China; (b) the ground truth image; (c) the result of FLICM; (d) the result of HMRF-FCM; (e) the result of RICM; and (f) the result of RFLICM.
In Figure 3(a), there are mainly three kinds of land cover types: the farmland, trees, and buildings. All of these types have rich texture information. The segmentation results of FLICM and HMRF-FCM have a lot of misclassifications. This is caused by the fact that using pixel-level interactions only cannot effectively model the complex texture pattern over the image. Furthermore, both acc and kc are very low. The RICM algorithm can get better results than these pixel-based methods but still has obvious misclassification. The RFLICM algorithm performs much better than the others since large scale (spectral, spatial, and structural) information can be employed in a fuzzy way in the clustering procedure. Therefore, it leads to a consistent result, with acc and kc being the highest, that is, more than 88% and 81%, respectively.
In Figure 4(a), four different kinds of land cover types are obviously distributed, with highly staggered appearance. The rich texture characteristics of this image make the results of FLICM and HMRF-FCM with serious misclassification. The result of RICM also contains much nonhomogeneous regions. It is mainly because that RICM cannot use the smooth term in a fuzzy way. Compared with the referenced methods, our proposed method shows a much better performance, with acc and kc more than 87% and 81%, respectively.
We also evaluate the computational time for all methods in the previous experiment. All of these algorithms are executed on a computer with an Intel Pentium Dual-Core 2.2 GHz central processing unit (CPU) and 4 G random access memory. The computation time is recorded in Table 5. It can be shown that the proposed method is a little slower than FLICM. It is slower mainly because of the time consumed by creating the initial oversegmentation and the use of more complicated distance function in our proposed method. But considering the accuracy obtained, it is worthy for us to pay such a price.
Computational time (in seconds).
Methods
FLICM
HMRF-FCM
RICM
RFLICM
Figure 1(a)
22.31
197.61
34.22
33.43
The 1st of Figure 2
29.78
180.72
46.18
39.62
The 2nd of Figure 2
21.9
213.74
32.10
29.61
The 3rd of Figure 2
58.35
126.50
61.72
62.84
The 4th of Figure 2
46.03
147.78
55.34
53.39
Figure 3(a)
40.95
85.26
50.99
48.55
Figure 4(a)
65.11
157.83
71.07
70.16
Average
40.63
158.50
50.23
48.23
5. Conclusion
In this paper, a novel region-level fuzzy local information c-means (RFLICM) algorithm for image segmentation was presented. The proposed method can detect the clusters of color texture images. It overcomes the disadvantages of the FLICM algorithm by incorporating region-level spectral, spatial, and structural information. The region-level fuzzy factor used in RFLICM aims to model in the fuzzy manner the large scale interactions between image regions instead of pixels. In terms of accuracy, experimental results of both synthetic and real datasets have illustrated that our proposed algorithm has better performance than that of FLICM, HMRF-FCM, and RICM. The color texture image experiments exhibited several unacceptable failures for all referenced methods, but the proposed RFLICM always gave accurate segmentation results.
However, compared with FLICM, RFLICM need a prepartition step which should provide a deliberate oversegmentation to preserve as much details as possible. Besides, it also needs to provide an initial value β(0) for the weighting parameter, while FLICM is completely free of any parameter determination. Although the segmentation results are not too sensitive to this value if it is in a proper range, we have to use the trial-and-error method to find a suitable value for the segmented images (in our experiments 10 is good enough).
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported in part by the National Natural Science Foundation of China under Grant no. 41001251, the China Scholarship Council (CSC) during a visit of Guoying Liu to the University of New Brunswick, the Canada Research Chairs Program and National Key Discipline of Forest Management support from Southwest Forestry University under Grant no. XKZ200901, the Key Technology Projects of Henan province of China (no. 132102210212), the Key Technology Projects of the Educational Department of Henan Province of China (no. 13A520011), and the Excellent Youth Teacher of Henan Educational Department of China (2011).
BezdekJ. C.PhamD. L.Spatial models for fuzzy clusteringChenS.ZhangD.Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measureBilginG.ErtürkS.YıldırımT.Unsupervised classification of hyperspectral-image data using fuzzy approaches that spatially exploit membership relationsHeL.GreenshieldsI. R.An MRF spatial fuzzy clustering method for fMRI SPMsXiaY.FengD.WangT.ZhaoR.ZhangY.Image segmentation by clustering of spatial patternsAhmedM. N.YamanyS. M.MohamedN.FaragA. A.MoriartyT.A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI dataChatzisS. P.VarvarigouT. A.A fuzzy clustering approach toward Hidden Markov random field models for enhanced spatially constrained image segmentationKrinidisS.ChatzisV.A robust fuzzy local information c-means clustering algorithmLinL.ZhuL.YangF.JiangT.A novel pixon-representation for image segmentation based on Markov random fieldYangF.JiangT.Pixon-based image segmentation with Markov random fieldsYuQ.ClausiD. A.IRGS: image segmentation using edge penalties and region growingQinA. K.ClausiD. A.Multivariate image segmentation using semantic region growing with adaptive edge penaltyYuP.QinA. K.ClausiD. A.Unsupervised polarimetric SAR image segmentation and classification using region growing with edge penaltyPragueThe prague texture segmentation datagenerator and benchmarkhttp://mosaic.utia.cas.cz/BesagJ.On the statistical analysis of dirty picturesGemanS.GemanD.Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of imagesComaniciuD.MeerP.Mean shift: a robust approach toward feature space analysisUnnikrishnanR.HebertM.Measures of similarityProceedings of the 7th IEEE Workshop on Applications of Computer Vision (WACV '05)January 200539440010.1109/acvmot.2005.712-s2.0-35348895982