Data Fusion and Fuzzy Clustering on Ratio Images for Change Detection in Synthetic Aperture Radar Images

The unsupervised approach to change detection via synthetic aperture radar (SAR) images becomes more and more popular. The three-step procedure is the most widely used procedure, but it does not work well with the Yellow River Estuary dataset obtained by two synthetic aperture radars.The difference of the two radars in imaging techniques causes severe noise, which seriously affects the difference images generated by a single change detector in step two, producing the difference image. To deal with problem, we propose a change detector to fuse the log-ratio (LR) and the mean-ratio (MR) images by a context independent variable behavior (CIVB) operator and can utilize the complement information in two ratio images. In order to validate the effectiveness of the proposed change detector, the change detector will be compared with three other change detectors, namely, the log-ratio (LR), mean-ratio (MR), and thewavelet-fusion (WR) operator, to deal with three datasetswith different characteristics.The four operators are applied not only in a widely used three-step procedure but also in a new approach. The experiments show that the false alarms and overall errors of change detection are greatly reduced, and the kappa and KCC are improved a lot. And its superiority can also be observed visually.


Introduction
Change detection [1][2][3] plays an important role in many applications [4][5][6][7][8][9][10][11] in our life, such as environmental monitoring, agricultural surveys, and damage assessment.The main task of change detection is to detect the change areas between several coregistered images taken at different times about the same place.With the rapid development of the remote sensing, the synthetic aperture radar (SAR) images are widely used for some unique advantages, but the inherent defect, known as speckle noise [12], heavily affects the accuracy of the results of the change detection.
The supervised approach and the unsupervised approach are the two main approaches to the change-detection problem.Though the ground truth (the reference image) is necessary in the supervised approach and not always available in many situations, the unsupervised change detection becomes more and more popular and many techniques are proposed.The three-step procedure [13] is the most widely used technique for its effectiveness and the three steps are about (1) image preprocessing, (2) producing the difference image (DI), and (3) analyzing the image to produce the change-detection map (CM).Besides the geometry and coregistration, in order to reduce the effect of the speckle noise, a filtering step is always necessary in the first step.Many filtering algorithms [14][15][16] are proposed, but the filtering process will result in missing too many details at the same time.The following step is about producing the difference image by comparing the two coregistration images pixel by pixel using a change detector.Two of the most common change detectors are the ratio detector and the difference detector, respectively.The former behaves better due to the multiplicative nature of speckle noise.The ratio detector is typically expressed in a logarithmic [17] or mean [18] scale, producing the log-ratio (LR) operator and the mean-ratio (MR) operator.Either of the two ratio operators has its own advantages and disadvantages [19].So the idea to make full use of the two ratio images obtained from the two ratio operators is attractive and promising.In the last step, a change-detection map is obtained by analyzing the difference image.The image analysis process in fact can be considered as an image segmentation process, which produces a binary image.The pixels in the change-detection map are assigned to the changed class and the unchanged class by two commonly used image segmentation methods, namely, the decision thresholds and the clustering algorithms.Due to the complexity of the difference image, the clustering algorithms are more widely used instead of the decision thresholds [20][21][22][23][24].And many clustering algorithms [25][26][27][28][29], based on the traditional Fuzzy -means (FCM) algorithm [30], are proposed with great performance.
The three-step procedure does show great performance and can be further improved by using other more effective filtering algorithms or clustering algorithms with better robustness.But it does not always work well with some datasets, such as the dataset at the region of the Yellow River Estuary.It is characterized by the different looks, which means that the two images are affected by the speckle noise in different levels and that the process to detect the changes becomes more difficult.Besides the severe noise in different levels, the changes in the different geographic sites with dissimilar types also increase the difficulty to detect them in the dataset at the region of the Yellow River Estuary.The analysis of the experiments will explain the problems in detail in Section 3. In 2005, a new approach [31] was proposed for change detection, which avoids filtering the original images and can retain as many details as possible since it is a detailpreserving and scale-driven method.We think it is a really good method to extract the homogeneous regions in the image and the new approach gives us inspiration to deal with the specific dataset.
In this paper, we propose a new change detector to produce the difference image.The new change detector is realized by fusing the log-ratio image and the meanratio image in a pretty simple but effective way and can utilize the complement information in two ratio images.By applying the new change detector, we can obtain the difference image with better performance.In order to validate the effectiveness of the proposed change detector, we will first apply the proposed change detector into the widely used three-step procedure.Then, in order to deal with the dataset at the region of the Yellow River Estuary, we propose a new change-detection approach which is another changedetection approach used in the experiments.The proposed change-detection approach is different from the commonly used three-step one, reflecting in two aspects.One is that the filtering process is not used, and another is that a process of extraction of the homogeneous regions is adopted before analyzing the obtained difference image.The changedetection map will be produced in the two ways as described above in our experiments.And the experiments show that the false alarms and overall errors of the change-detection results are both greatly reduced.The effectiveness and superiority of the proposed change detector are both validated by both visual and quantitative experiments results.We can also see that the proposed change-detection approach does perform well to the datasets with better change-detection results.
The rest of this paper is organized in three sections.Section 2 gives a detailed description about the proposed Filtering preprocess Filtering preprocess

Change-detection detector
Analysis of the images to produce the CM Figure 1: Flowchart of the widely used three-step procedure.

Extraction of homogeneous regions
Analysis of the images to produce the CM Change-detection detector Figure 2: Flowchart of the proposed change-detection approach.
method.Section 3 is about the experiments study.The conclusions are presented in Section 4.

Proposed Method
Let us consider two coregistered intensity SAR images,  1 and  2 , acquired over the same area at two different times,  1 and  2 .The two images are of the same size,  × .Here, we denote the two images by  1 = { 1 (, ), 1 ≤  ≤ , 1 ≤  ≤ } and  2 = { 2 (, ), 1 ≤  ≤ , 1 ≤  ≤ }, respectively.The difference image produced by the two images is very important and we want to get the difference image with less speckle noise and more detailed information.As mentioned in Section 1, the change detector will be applied in two ways to deal with the change-detection problem.The flowcharts of the two ways to deal with the change-detection problem are shown in Figures 1 and 2, respectively.Figure 1 is the flowchart of the widely used three-step procedure described in Section 1 in detail, while Figure 2 is the flowchart of the second approach in order to deal with the dataset at the region of the Yellow River Estuary and the corresponding details will be described in the next part.
2.1.The Proposed New Change Detector.As mentioned in Section 1, the ratio operator performs better than the difference operator and is always expressed in a logarithmic or mean scale, producing the log-ratio (LR) operator and the mean-ratio (MR) operator.Either of the two ratio operators has its own advantages and disadvantages.The two ratio operators can produce two ratio difference images with different information about the changes.So to get the optimal difference image by utilizing the information in two ratio images is a really exciting idea.In [19], a new change detector based on the two ratio images is proposed to produce the difference image.It is based on wavelet fusion and the wavelet coefficients for low-frequency and high-frequency bands are fused by rules based on weight averaging and minimum standard deviation, respectively.The fusion rules can be described as follows: where  represents the mean-ratio image and  represents the log-ratio image. represents the new fused image. LL means the low-frequency coefficients.Three high-frequency coefficients at point (, ) in the corresponding subimages are denoted by   (, ), ( = LH, HL, HH).  (, ) represents the local area energy coefficient and is described as follows: where  , means the local window centered on (, ) and   () denotes the value of the th wavelet coefficient that is around the local window.We can see that the fusion is realized by fusing the wavelet coefficients of low frequency and high frequency separately.The coefficients are obtained by applying the discrete wavelet transform (DWT) to the two ratio images.It is natural to think about some other multiscale transforms instead of the discrete wavelet transform to fuse the log-ratio image and the mean-ratio image, such as the curvelets and contourlets, but here we want to combine the two ratio images to obtain the complement information in a simpler and more effective way.The log-ratio image is obtained by applying the log-ratio operator to the two coregistered intensity SAR images:  1 and  2 .The log-ratio operator is defined as follows: where   represents the log-ratio image.
In a similar way, the mean-ratio image, represented as   , can be obtained by applying the mean-ratio operator instead of the log-ratio operator.The mean-ratio operator is formulated as where  1 (, ) and  2 (, ) represent the local mean values of the pixels involved in a neighborhood of point (, ) in the two SAR images  1 and  2 , respectively.
Here we want to combine information from two sources, namely, the log-ratio image and the mean-ratio image, and it is exactly the main task of data fusion.In 1996, a paper [32] titled "Information combination operators for data fusion: a comparative review with classification" was published, introducing the information combination operators for data fusion in a systematic way.It provides a guide for choosing an operator in a given problem, which depends on the desired properties of the operators.In this paper, three classes of the fusion operators are defined.Let  and  denote two pieces of information to be combined.And a function  acting on  and  defines a fusion operator.The first class, called context independent constant behavior (CICB) operators, is composed of operators which have the same behavior whatever the values of the information to combine and which are computed without any contextual or external information.The second class is composed of operators which are context independent like in the first class but whose behavior depends on the values of  and .The operators in this class are called context independent variable behavior (CIVB) operators.The third class, context dependent (CD) operators, is composed of operators which not only are from  and  but also depend on global knowledge or measure of the sources to be fused (like conflict between sources or reliability of sources).
Considering our problem, we would like to choose a fusion operator which is context independent without considering the global knowledge or measure of the sources to be fused.And due to the fact that the fusion operator cannot be guaranteed to have the same behavior whatever the values of the information to combine, a fusion operator in the second class is more appropriate to our problem, which belongs to the context independent variable behavior (CIVB) operators.Here the information means the intensity in SAR images, so the two pieces of information  and  mean the intensity in the two ratio images, respectively.The two pieces of information in two ratio images can be both normalized into [0, 1].The intensity in the fused image is 0 when the intensities in the two images of the same pixel are both 0, and the intensity in the fused image is 1 when the intensities in the two images of the pixel are both 1.The conditions of symmetrical sums are satisfied in the paper.Here the chosen fusion operator is expressed as Using   and   instead of  and  in (5), we will get the new change detector which is formulated as where   represents the fusion image.

The Extraction of the Homogeneous Regions.
In 2005, a new approach [31] was proposed for change detection, which avoids filtering the original images and can retain as many details as possible since it is a detail-preserving scaledriven method.It is a really good method to extract the homogeneous regions in the image.
The process is based on three main steps: (1) multiscale decomposition of the DI, (2) a selection of the reliable scales for each pixel, and (3) a scale-driven combination of the selected scales, and we choose to fuse all reliable scales at the feature level for its good performance.The first step includes multiscale decomposition of the DI and a reconstruction only considering the approximation subbands which produces a set of images representing the change information.The key of the following step is the adaptive scale identification and deserves a detailed description because of its importance.For a pixel of the spatial position (, ) with a given scale , to determine whether it belongs to a border or a homogenous area is not easy.Here, it is realized by comparing the local coefficient of variation and global coefficient of variation of the pixel.The local coefficient of variation (LCV  ) proposed in [31] and the global coefficient of variation (CV  ) are formulated as LCV  (, ) =   (, )   (, ) , where   (, ) represents the local standard deviation,   (, ) represents the local mean,   represents the global standard deviation, and   represents the global mean.Here  = 0, 1, . . .,  − 1.The resolution scale 0 is the original image to reserve more information and  − 1 is the largest resolution scale.The scale can be considered as reliable only when ( 8) is satisfied: But there is a problem that the information at low-resolution levels is not so reliable since the scales details and edge information have been removed in the previous step.So a scale  for a given pixel in a resolution level is defined as reliable only when all the scales 0, 1, . . .,  − 1 are judged as reliable; in other words, all the scales 0, 1, . . .,  − 1 for a given pixel are satisfied (8).In the last step, we average the intensity of the pixels to extract the homogeneous regions.The approach to extract homogeneous regions is characterized with good performance due to the different trade-off between speckle reduction and preservation of geometrical details.

Automatic Analysis by FCM S1.
To improve the performance of the FCM, many clustering algorithms [25][26][27] are proposed, which are robust to the noise for incorporating the local spatial and gray-level information.The FCM S [25] is one of them and is proposed by Ahmed et al., but the computational complexity increases.To deal with this problem, two variants, FCM S1 and FCM S2 [27], are proposed, which are realized by introducing the extra mean-filtered image and median-filter image to replace the neighborhood term, respectively.The objective function, the calculation of membership partition matrix, and the cluster centers are defined as ( 9), (10), and (11), respectively: where ‖ ⋅ ‖ stands for the Euclidean norm,  is the number of the clustering,  is the total number of the pixels,   is the gray level value of the th pixel,   is the mean value of the gray level values of the neighboring pixels falling into the window around   ,   stands for the prototype value of the th center,    is the fuzzy membership of the th pixel with respect to the th cluster,  is a fuzzy exponent, and the parameter  is used to control the trade-off between the original image and its corresponding mean-filtered image.The parameter is artificial and is always chosen by experience or by using a trial-and-error method.

Experimental Results
In order to assess the effectiveness of the proposed change detector, we adopt three datasets with different characteristics in the experiments.The proposed new change detector will be compared with other three change detectors in two ways.The three change detectors are the log-ratio (LR), mean-ratio (MR), and the wavelet-fusion (WR) operator [19], respectively.The proposed new fused ratio operator is short for FR.The four change detectors are applied in two ways, and the flowcharts of the two ways are shown in Figures 1 and 2, respectively.We will also introduce the comparison experiments as well as some evaluation criteria of the experiments results in this section.

Dataset Description.
The first dataset, the Bern dataset, is a section (301 × 301) of two SAR images (see Figures 3(a) and 3(b)) acquired by the European Remote Sensing 2 satellite SAR sensor over an area near the city of Bern, Switzerland, in April and May 1999, respectively.Because parts of the cities of Thun and Bern and the airport of Bern entirely are flooded by River Aare between the two dates, the Aare valley between Bern and Thun was selected as a test site for detecting flooded areas.Figure 3(c) is the available ground truth image (the reference image) which is obtained by integrating prior information with photo interpretation.
The last two datasets are both from the same SAR images, which are about the region of the Yellow River Estuary over Dongying in Shandong province, China.Particularly, the image taken on June 18, 2008, is four-look data while the image on June 19, 2009, is one-look data.The character leads to the huge difference of the speckle noise level between the two images, increasing the difficulty to detect changes.Because the original size of the two images is as large as 7666 × 7692, it is difficult for us to acquire the reference image.So we select two typical areas with different characters to produce the third and the fourth datasets, namely, the farmland dataset and the coastline dataset.
In detail, the second dataset is about the farmland dataset and the size of the images is 291 × 306.The two images are shown in Figures 4(a) and 4(b), respectively.We can see that a block of farmland is landlocked and the changed regions are relatively large and regular.The reference image is shown in Figure 4(c).And the third dataset is the coastline dataset, shown in Figures 5(a) and 5(b).It is a section of an inland water area and the size of the images in this dataset is 280 × 450.The changed areas are concentrated on the borderline of the river and only a small part of the image is changed pixels (1075 changed pixels and 124925 unchanged pixels in  the ground truth map), which is comparatively hard to detect.The reference image is shown in Figure 5(c).

Evaluation Indexes.
Besides the visual inspection, a quantitative analysis of the change-detection result is taken.It is realized by comparing the change-detection map to the corresponding reference image and there are some quantitative indexes [33].FN is short for the false negative, which means the changed pixel that was not detected.FP is short for the false positive, which means the wrongly detected as changed pixel.We should also pay attention to true positive and true negative, written as TP and TN for short, which means correctly detected as changed and unchanged pixel, respectively.So we can get the overall error (OE) and the percentage correct classification (PCC), which are formulated as OE = FNs + FPs, PCC = (TP + TN) And there are more indexes when evaluating the obtained change-detection results, such as the kappa coefficient [34] and the receiver-operating-characteristics (ROC).The value of the kappa coefficient is between 0 and 1, reflecting the accuracy or agreement between the change-detection map and the reference image.When the final binary map is coincided completely with the reference image, the value of the kappa coefficient is equal to 1.The ROC plots the detection probability versus the false alarm probability to determine a desire level of performance.

Results of the Bern Dataset.
The overall description of the comparative experiments is at the beginning of Section 3 and here are some complement explanations.The experiments are noted as LR 1, MR 1, WR 1, FR 1, LR 2, MR 2, WR 2, and FR 2, respectively, for short.The number "1" means the three-step procedure while the number "2" represents the new approach [31] (see Figures 1 and 2).The FCM S1 method is applied to produce the change-detection map and the parameter  is set to 1.8 with better performance by using a trial-and-error method.In this part, the experiment results of the Bern dataset are presented and the ones of the other two datasets will be presented with the same symbolic signs.
The difference images generated by the different operators in two ways are shown in Figure 6 and the corresponding change-detection maps are shown in Figure 7.In Figure 6, we can see that the FR operator performs best for the clearness of the outline of the changed regions and the robustness to the speckle noise.The MR operator behaves worst, and the LR and WR operators represent equal results.Figure 8 is the ROC plots and we can see that the ROC curve obtained by FR 2 outperforms the other ROC curves obtained by the other methods.The quantitative results of the changedetection maps are shown in Table 1 and are consistent with the results of the visual analysis.When comparing the quantitative results obtained by LR 1, MR 1, WR 1, and FR 1, we can see that the values of the FP and OE produced by the FR operator are both smallest.The values of PCC and kappa coefficient produced by the FR operator are both largest.And, comparing the quantitative results obtained by LR 2, MR 2, WR 2, and FR 2, we can see the same situation.The results do validate the effectiveness of the proposed change detector.Even though the results obtained by the FR in the proposed change-detection approach is not better than the results obtained by the FR in the widely used three-step approach, the results using the proposed change-detection approach perform better than the other changed detectors in both ways.

Results of the Farmland Dataset.
The difference images generated by the different operators in two ways are shown in Figure 9 and the corresponding change-detection maps are shown in Figure 10.The FR and the LR operators have better visual performance.Moreover, the difference between the changed and unchanged classes in the results of FR and LR is more obvious than in the other ones.From the ground truth image in Figure 4(c), we can see that the changed regions are relatively large and regular.Every two small strip changed regions are too close, increasing the difficulty to detect the unchanged part between them.And the two irregular changes in the lower right corner are easy to be overwhelmed in the noise.The borderline of the strip changed regions is highly blurred by the WR [11] due to the fact that the wavelet coefficient of high frequency is not completely fused.The ROC plots are shown in Figure 11 and we can see that the ROC curve obtained by FR 2 outperforms the other ROC curves obtained by other methods.The quantitative analysis results of the change-detection maps are shown in Table 2 and are consistent with the results of the visual analysis.The unchanged class in the difference image produced by FR is more homogeneous, reducing the number of the FP effectively.Even though the value of the FN is slightly higher, the change-detection map obtained by the FR has the smallest value of OE and the highest value of PCC.And the kappa coefficient obtained by the proposed method with the proposed change detector is significantly better than the others.

Results of the Coastline Dataset.
The coastline dataset is more difficult to detect the changes than the last farmland dataset for some reasons.The changed regions in this dataset are several fairly small parts, which are easy to be overwhelmed in the serious speckle noise.And the unchanged small region inside the cyclic changed region is really a challenge to detect.The same comparative experiments are set for proposed method based on extracting the homogenous regions to detect changes is really suitable to deal with the dataset with large homogenous regions and severe noises.The effectiveness and superiority are validated by both visual and quantitative results.

Conclusion
In this paper, we propose a new change detector to fuse the log-ratio (LR) image and the mean-ratio (MR) image by a context independent variable behavior (CIVB) operator.The fusion rule is simple but quite effective.And the proposed change detector utilizes the complement information in two ratio images.In this paper, the new proposed change detector is applied in not only a widely used three-step procedure but also a new approach to deal with the dataset at the region of the Yellow River Estuary via SAR images due to its specificity.The new approach adopts a detail-preserving scale-driven approach to extract the homogeneous regions and is suitable to deal with the dataset with large homogenous regions and

Figure 3 :Figure 4 :
Figure 3: The Bern dataset: (a) image acquired in April 1999 and (b) May 1999 and (c) ground truth image.

Figure 11 :
Figure 11: ROC plot comparison between the four change detectors on the farmland dataset.

Table 1 :
Change-detection results of the first dataset.

Table 2 :
Change-detection results of the second dataset.