Research on Multifeature Segmentation Method of Remote Sensing Images Based on Graph Theory

According to the characteristics of high-resolution remote sensing (RS) images, a new multifeature segmentation method of highresolution remote sensing images combining the spectrum, shape, and texture features based on graph theory is presented in the paper. Firstly, the quadtree segmentation method is used to partition the original image. Secondly, the spectrum, shape, and texture weight components are calculated all based on the constructed graph. The matching degree between pixels and the texture is computed similarity. Finally, the ratio cut standards combination of the spectrum, shape, and texture weight components is used for the final segmentation. The experimental results show that this method can obtain more ideal results and higher segmentation accuracy applied to RS image than those traditional methods.


Introduction
In the past decade, many scholars have done the research about graph-based approaches to image segmentation.Graph theory has got a lot of attention because of its representational power and flexibility properties.Bai et al. [1] apply graph cut (GC) theory to the classification of hyperspectral remote sensing images.Fuzzy SVM classifier and the GC-based classification were used in two-step classification strategy in this paper.Felzenszwalb and Huttenlocher [2] define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation.The time complexity of the method runs in ( log ) time for  graph edges.The work of Wang and Siskind [3] presents cut ratio as a new cost function of graph methods for segmenting image.This method is useful for some image segmentation applications.Cui and Zhang [4] use Minimum Span Tree optimal theory to realize object based on high-resolution image segmentation.The result proved that this method can obtain high quality segmentation.Kato et al. [5,6] propose a Markov random field (MRF) image segmentation model based on the integration of colour and texture descriptors.This method can use both synthetic and natural color images.
The objectives of this paper are to obtain better image segmentation results and relative high segmentation accuracy for high-resolution RS images.For RS image, the algorithm's time complexity and space complexity rate will be high if only graph theory for image segmentation is used.Quarter-tree segmentation method is a fast image segmentation algorithm, but it cannot divide meaningful target area for RS images.If the threshold is set too low, oversegmentation phenomenon will be very serious.If the threshold is set too high, it cannot form a more accurate target edge [1,13].Therefore, this paper designs a new method by combining the merits of quarter-tree segmentation and ratio cut (-cut) algorithm, and the method can be used in high-resolution RS images.This method is effective to reduce the size of the graph vertices, improving the accuracy of image segmentation.It first establishes the mapping relation of RS image and graph and then sets an energy function of the graph according to the remarkable weights.We can solve the energy function to get the minimum which will lead to the result of graph segmentation.At last we mapped the graph segmentation result back to image.Because the construction of graph and the extraction of remarkable weights can be based on both pixel and image blocks, the methods based on graph theory will be good in image segmentation.
The organization of this paper is as follows.In the second section, quadtree segmentation method and -cut theory for multifeature segmentation of RS image are described.Results and discussion are given in Section 3. The final section is the conclusions.

The 𝑅-Cut Theory for Multifeature
Segmentation of RS Image

Quadtree Segmentation Method.
A quadtree is a tree data structure in which each internal node has exactly four children.Quadtrees are most often used to zone a two-dimensional space by recursively subdividing it into four quadrants or regions [16].Quadtree decomposition is currently a valuable method in image processing and computer graphics.The procedure of quadtree segmentation is as follows.
Step 1.The original image (typically 2 × 2) is divided into four same size regions.
Step 2. It is to detect the constant gray level of each region segmentation image.
Step 3. If it cannot meet the request of constant gray level of the image, then each district will be divided into four areas of the same size and go to Step 2.
Step 4. If it meets the request, then stop the iterative process.

𝑅-Cut
Standards.The graph partition problem is defined on data represented in the form of a graph  = (, ), with  vertices and  edges.Where in form,  corresponds to the image, the vertices correspond to regions, and the edges correspond to adjacent relations between the regions.Ratio cut represents the ratio of the corresponding sums of two different weights of edges along the cut boundary.A minimum ratio cut refers to the smallest cut ratio [3].
The energy function of -cut standards is as follows: where  and  represent two different image blocks, respectively, and  1 (, ) and  2 (, ) calculated the energy of cut sets by using two methods of weight calculation, respectively: where  denotes the vertex of , V denotes the vertex of ,  1 (, V) and  2 (, V) denote the weights associated with each edge (, V), respectively,  1 (, V) is the first edge weight, and  2 (, V) is the second edge weight.

𝑅-Cut
Reduction Algorithm.The -cut reduction algorithm steps are as follows.
Step 1 (calculate minimum ratio cut).In order to simplify the calculation, we calculate the minimum ratio ring instead of the minimum ratio.The dual graph   = (  ,   ) of the graph  is constructed and shown in Figure 1.There is a oneto-one correspondence between the minimum ratio cut of graph  and the minimum ratio ring of the dual graph   , therefore, the problem that calculate the minimum ratio cut set { 1 ,  2 , . . .,   } of the original  can be transformed into the minimum ratio ring of the dual graph   .
Step 2 (calculate the minimum ratio ring).In order to simplify the calculation, the negative simple ring is computed instead of the minimum ratio ring.The   () representations of the dual graph   with the linear conversion    11 , where    11 is obtained by doing the conversion:   11 =   1 −   2 and functions   1 and   2 are the edge weights function of graph   .The conversion of weights function will not change the minimum ratio ring loop of the dual graph   , so it will not change the minimum ratio cut of the original .Only the minimum loop cost of   ( * ) is nil consideration, and the graph   has a minimum ratio of the loop which contains the loop ratio  * .The graph   has a minimum ratio of the loop which contains the loop ratio  * , if and only if the minimum loop cost of   ( * ) is equal to zero.The relationship between  * and  is as follows.
If   () has a negative cost loop, then  * < .While if   () does not have negative cost loop, then  * ≥ .Let  min and  ax , respectively, be the minimum and maximum loop ratio of , so  min =  min and  max =  ax .Then,  min ≤  * ≤  max .Let  =  min +  max /2; if   () has a negative cost loop, then  max value is set to ; otherwise the  min value is set to , continuing the repeated calculation, until we cannot find a negative cost of simple loop concerning the one corresponding to ; and now  is the minimum loop ratio  * and at the same time the negative cost simple loop of   () is the negative cost simple loop which we want to find.
Step 3 (calculate the minimum cost perfect matching).In order to reduce the calculation, the negative simple ring is computed instead of the minimum cost perfect matching.(ii) For each edge (  , V  ) of graph   , graph   contains two corresponding vertices   V and V   and five corresponding edges.Figure 2 shows the weight value of the five edges.
According to   which is obtained through the above three steps, we can calculate the  1 (, ) and  2 (, ) and obtain the minimum ratio cut according to formula (1).

Weight Calculation.
In this paper, a multifeature segmentation method which takes into account the spectrum, shape, and texture features of RS image is applied.
The weight component based on the spectrum is defined as  spectrum  , the weight component based on the shape is defined as  shape  , the weight component based on the texture is defined as  texture  , and   is the combination of the above three aspects of information [8]: The weight component based on the spectrum In formula (4),  color denotes the standard deviation of the pixel color. 2 can be described as follows: where  is the number of filter's types.If  and  correspond to two pixels,  2 is used to record the spectral similarity between the pixels  and .If  and are corresponding to the two blocks,  2 is used to record the spectral similarity between the blocks  and .
The weight component based on the shape where max ∈   con () is used to mark the matching degree between the two pixels or blocks  and , which is obtained by calculating the maximum value of all the probability coefficient along the line  of the set of pixels   after connecting  and  in a straight line.If this line exactly intersects with a profile, then max ∈   con () is large, the weight is small, and  and  may belong to two classes; on the contrary, if the line is parallel to the profile, then max ∈   con () is small, the weight is greater, and  and  may belong to the same class.
The weight component based on the texture  texture  is where ℎ  and ℎ  are the histogram obtained by doing texture operator transform for  and , respectively, and  texture denotes the standard deviation of the texture of object. 2 is used to record the texture similarity between  and .If the difference between ℎ  and ℎ  is too large, the values of  2 will be large, and  texture  is very small.So  and  do not belong to the same class [1][2][3][4]8].
The algorithm flowchart is shown in Figure 3.

Evaluation Method of Segmentation Results
. Evaluation method of image segmentation is divided into qualitative and quantitative analysis.This paper makes a quantitative analysis on the segmentation results by using the theory proposed in [14,[17][18][19][20].The specific evaluation index is defined as follows.
3.1.1.Homogeneity.From [14], we handle the standard deviation of all the pixels as a measure of the object homogeneity criterion.The standard deviation of the object can be written as where  is the number of all pixels within the object,   represents the pixel gray value of pixel , and  represents the gray mean of the object.

Heterogeneity.
For each object, we calculate the average difference absolute value of the object with the neighborhood to reflect the degree of difference between the object and the adjacent object [14].The formula of heterogeneity can be written as where  is the boundary length of the current object,   is the common edge length of the current object with  adjacent objects,  is the gray mean of the current object,   is the gray mean of  adjacent objects, and  is the number of adjacent objects with the current object.

Segmentation Evaluation Index (SEI). SEI is inversely
proportional with its homogeneity and proportional with its heterogeneity [14].The SEI of the object is defined as follows:

Probabilistic Rand Index (PRI).
The Probabilistic Rand Index (PRI) counts the similarity of pairs of pixels whose labels are consistent between the computed segmentation and the ground truth.The expression of PRI can be defined as This measure takes values in [0, 1]-0 when two images have no similarities, and  = 1 when two images are identical, where  test is the segmentation that is to be compared with the reference segmentation image and {  } is ground-truth segmentations, where   denotes the event of a pair of pixels  and  having the same label and   its probability [17][18][19].

The Variation of Information (VoI)
. The Variation of Information (VoI) metric defines the distance between two segmentations as the average conditional entropy of one segmentation given the other one and thus roughly measures the amount of randomness in one segmentation which can be explained by the other [20].The formula of VoI can be written as where  and , respectively, represent the entropies and the mutual information between two clustering of data  and data   .This measure takes values in [0, 1].

The Global Consistency Error (GCE).
The Global Consistency Error (GCE) measures the extent to which one segmentation can be viewed as a refinement of the other.GCE can be defined as 1 and  2 are input segmentations images.( 1 ,  2 ,   ) and ( 1 ,  2 ,   ) are the local refinement error, respectively.( 1 ,  2 ,   ) is zero precisely when  1 is a refinement of  2 as pixel   , but not vice versa.[21].

The Experiment Results.
In this section, we apply the proposed algorithms to real high-resolution data by the ALOS high-resolution RS images of Shi Zuishan Industrial Park, Ningxia, China.Its ground spatial resolution is 2.5 m and the size is 512 × 512 pixels.According to the human visual, field surveys and spectral measurement results, we select five samples, lime pile, cinder heap, house, road, and wasteland.To validate the algorithm, the images were segmented from the spectral, shape, texture, and multifeature segmentation based on graph theory, respectively, and then made a comparison among the four segmentation results.
Original image and various algorithms segmentation results are shown in Figures 4 and 5, respectively.From Figure 4, the spectral segmentation based on graph theory has certain limitation; it is prone to split too small for textured areas, but it is less likely to split for the areas of relatively close texture.The shape segmentation based on graph theory is prone to split too small.Although the method of the texture segmentation based on graph theory can get a better segmentation of all types of surface features, it is not obvious to the boundary between the surface features.The multifeature segmentation method of remote sensing images based on graph theory not only can make the measurement, the spectra, and texture information of different objects better, reflect the differences between the different types of surface features, and achieve better segmentation, but also can accurately obtain the boundary between different types of objects; in short, it can ensure the accuracy of subsequent analysis.To be more accurate and objective evaluation of segmentation results of the algorithm, this paper uses the abovementioned evaluation method for quantitative evaluation of the segmentation results.The lime heap and cinder heap are selected as evaluation object because the paper mainly monitors industry solid waste.Specific segmentation scale evaluation results of origin RS image 1 are shown in Table 1.
From Table 1, we can see that the lime heap homogeneity index is 2.1045 with the multifeature segmentation method of remote sensing images based on graph theory, which is smaller than the lime heap homogeneity index of spectralbased and shape-based segmentation, and the cinder heap homogeneity index is 0.9877, which is smaller than the cinder heap homogeneity index of texture-based segmentation and is close to the cinder heap homogeneity index of shapebased segmentation; this comparison and contrast herein prove that, by using the multifeature segmentation method based on graph theory, one can get a better measurement of the spectrum and texture information of surface features.The heterogeneity index of the multifeature segmentation based on graph theory is greater than the other three segmentation methods, and it further shows that the multifeature segmentation method based on graph theory can obtain more precise boundaries between different types of surface features.In summary, the results of the multifeature segmentation method based on graph theory make good internal object homogeneity, and at the same time there is an obvious contrast between adjacent objects.
The results obtained with the other segmentation methods and the proposed algorithm over two high-resolution RS images are shown in Table 2.The parameters of PRI, GCA, and VoI of each segmentation methods are computed.Quadtree, watershed, mean shift, multiresolution, and the proposed method are the region-based segmentation methods.The quadtree method starts at the root of the tree that represents the whole image.If it is found nonuniform (not homogeneous), then it splits into four son squares (the splitting process).If, in contrast, four son squares are homogeneous, they are merged as several connected components (the merging process).This process continues recursively until no further splits or merges are possible.The multiresolution approach partitions the image at different scale, using a pyramid or quadtree structure.The watershed approach considers the gradient magnitude of an image as a topographic surface.Pixels having the highest gradient magnitude intensities correspond to watershed lines, which indicate the region boundaries.Mean shift method is defined as finding modes in a set of data samples, showing an underlying probability density function.Canny operator and sobel operator are boundary-based segmentation method.
From Table 2, taking into account the quality of the results from the evaluation parameters, it will be noticed that the best results are reached by the proposed method.The value of PRI of this method is the highest compared with the other segmentation methods.This is mainly due to the fact that this method combines the spectrum, shape, and texture of image and the segmentation region is close to real region.From the results of Table 2, we can see that the results of region-based segmentation methods are better than the boundary-based segmentation methods.

Summary
In this paper, we took into account a number of feature information of the image and used -cut theory for RS images segmentation.Experimental comparison shows that multifeature segmentation method based on graph theory achieved better segmentation results than the methods based on single feature.Overall, the method can be used in highresolution RS images.Even though the method also has shortcomings, such as the effectiveness and the implementing speed of algorithms which are not very satisfactory, in the future, we will keep on seeking an efficient solving process and the weight calculation formula to apply to RS image segmentation.
Construct a new graph   = (  ,   ) from the graph   which is obtained by the previous step.Graph   contains negative cost loop, if and only if   has the minimum cost perfect matching.From graph   to graph   , the specific conversion principles are as follows.(i) For each vertex   1 of graph   , graph   contains two vertices   1 ,   2 and an edge (  1 ,   2 ) of which weight value is equal to zero. a

Table 2 :
Segmentation evaluation results over region-based algorithm and boundary-based algorithm.