Multivariate Self-Dual Morphological Operators Based on Extremum Constraint

Self-dual morphological operators (SDMO) do not rely on whether one starts the sequence with erosion or dilation; they treat the image foreground and background identically. However, it is difficult to extend SDMO to multichannel images. Based on the self-duality property of traditional morphological operators and the theory of extremum constraint, this paper gives a complete characterization for the construction of multivariate SDMO.We introduce a pair of symmetric vector orderings (SVO) to construct multivariate dual morphological operators. Furthermore, utilizing extremum constraint to optimize multivariate morphological operators, we construct multivariate SDMO. Finally, we illustrate the importance and effectiveness of the multivariate SDMO by applications of noise removal and segmentation performance.The experimental results show that the proposedmultivariate SDMO achieves better results, and they suppress noisesmore efficientlywithout losing image details comparedwith other filteringmethods. Moreover, the proposed multivariate SDMO is also shown to have the best segmentation performance after the filtered images via watershed transformation.


Introduction
One of the main issues for mathematical morphology is its extension to the multivariable case [1].So far, a lot of effort has been made to design multivariate morphological operators which can be applied in multichannel images [2][3][4][5].The key point of constructing morphological operators for multichannel images consists in the definition of the ordering scheme for multivariate data.Thus, researchers have proposed many vector orderings, which can be divided into four categories: marginal ordering, lexicographical ordering, partial ordering, and reduced ordering in which the lexicographical ordering is one of the most popular algorithms [6].Louverdis presented multivariate morphological operators based on lexicographical ordering in HSV color space [7].As lexicographical ordering is most suitable for the situations in which an order of "importance" exists on the available channels, the vector ordering based on the combinations of the different vector distances (e.g.Mahalanobis distance, Euclidean distance) and lexicographical ordering was proposed by Angulo [8].Recently, Aptoula et al. proposed -trimmed lexicographical extremum estimation algorithms, based on the new lexicographical ordering, and constructed the new multivariate morphological operators [9,10].In order to make further improvement on the performance of multivariate morphological operators for color image filtering and segmentation, Angulo studied quaternion properties and applied quaternion decomposition to color image representation and then proposed a new lexicographical ordering based on quaternion decomposition, which can provide better results in image processing [11].According to [10,11], Lei et al. proposed vector ordering based on fuzzy extremum estimation algorithm, and then constructed multivariate morphological operators.The proposed operators have a high performance on color image filtering [12].
In recent years, various complex mathematical tools, such as machine learning [13], rand projection depth function [14], principal component analysis [15], hypercomplex [16], probabilistic extrema estimation [17], and group-invariant frames [18], are employed to develop multivariate morphological theory and improve the performance on color image filtering and segmentation.Lezoray proposed nonlocal vector mathematical morphology.First, unsupervised vector ordering was proposed according to dictionary learning and machine learning.And then the improved multivariate morphological operators were proposed [19].Velasco-Forero studied multivariate mathematical morphology based on random projection depth function and proposed vector ordering algorithm based on P-ordering.The proposed algorithm solved the problem of extending morphological theory to multichannel images [14].
A lot of multivariate morphological operators have been proposed, but few researches on the properties have been reported.Most of the morphological operators occur in pairs of dual operators, such as erosion/dilation and opening/closing.In the binary case, erosion on the background of an image is equivalent to the dilation on its foreground [20].Also in the grayscale case, the complement of erosion on the original image is equivalent to dilation on its complement.The traditional morphological operators will lead to a heavy deviation of the gray values of filtered images [21].Therefore, the final results depend on whether erosion or the dual dilation is used as the first operator in the sequence.For example, images operated by those morphological operators of which the first operator is erosion will tend to be darker; in contrast, images operated by operators of which the first operator is the dual dilation will have the opposite results.In order to treat the image foreground and background identically, self-dual morphological operators can be applied in image processing [22].Bouaynaya et al. studied SDMO in the case of the general theory of lattice morphology and spatially variant morphology and illustrated the importance of the self-duality property by an application to speckle noise removal in radar images [23].Self-dual morphological operators have been widely used in binary and grayscale images.However, it is difficult to extend the applications of SDMO to multichannel images.
To address this issue, we study the existing multivariate morphological operators and find that vector ordering is the key factor for the duality.By defining a pair of SVO in RGB color space, we construct multivariate dual morphological operators and multivariate SDMO.However, it is well known that VMF (vector median filters) is able to provide better results than multivariate morphological operators for color image filtering.Therefore, multivariate SDMO based on extremum constraint are finally presented.The experimental results demonstrate that the proposed multivariate SDMO performs better than the popular VMFs.As image filtering is the foundation of image segmentation and understanding, we applied watershed transformation to the images filtered by various filters.The segmentation results show that the proposed multivariate SDMO also has the best segmentation performance.
The structure of this paper is organized as follows.Section 2 introduces the traditional self-dual morphological operators.Section 3 analyzes the reasons why the existing multivariate morphological operators are not dual and then gives the solution of the duality of multivariate morphological operators.Multivariate self-dual morphological operators based on extremum constrain are presented in Section 4.
Section 5 presents the experimental results and analysis, and Section 6 gives conclusions of this paper.

Self-Dual Morphological Operators
Before entering a general discussion on the construction of self-dual operators, the related concepts of self-dual morphological transformation are proposed.Definition 1.The complement of an image , denoted by   , is defined for each pixel h as the maximum value of the data type used for storing the image minus the value of the image  at position h: where h = {ℎ The median operation is a self-dual neighborhood image transformation since it replaces each pixel value by the median value of the original image pixels located in a window centered at the considered pixel.The detailed proof is proposed as below.For color images, Zanoguera and Meyer studied vector leveling which are self-dual vector morphological filers and proposed a convenient algorithm for calculating a nonseparable rotation-invariant vector leveling because the algorithm which derives from the definition is rather costly.The proposed algorithm is applied to integer images without compromising convergence issues and provides better results than those obtained when independently applying a scalar leveling to the three color components [24].
The traditional morphological filtering operators always have the problem that the filtered image is brighter or darker than the original image.Fortunately, self-dual morphological transformation can solve the problem; it can maintain the gray value of the filtered image.Based on dual and self-dual transformation, Heijmans proposed morphological median operators, which are also called self-dual morphological operators [22], to address the problem that the classical morphological filters are unfair for the foreground and background of an image.Proposition 4. Let {,  * } be a pair of dual transformation, and let  and  denote morphological erosion and dilation, respectively.Let  be an increasing transformation and let  * be the dual transformation of .Thus, morphological median denoted by  can be represented as follows: where  denotes identity operator.We can demonstrate that  is a self-dual transformation.
Self-dual morphological filtering operators are applied in image filtering, and the results are shown in Figure 1, where  is the self-dual filtering operators based on openingclosing and closing-opening and the structuring element Figures 1(b)-1(c) show that the image filtered by openingclosing turns darker, while the image filtered by closingopening turns brighter than before.Obviously, the conventional morphological filters lead to a drift of the gray value of filtered image.Self-dual morphological filters overcome the problem; it can keep the gray value of an image since self-dual morphological filters equally treat the foreground and background of the image as shown in Figure 1(d).

Multivariate Self-Dual Morphological Operators
We firstly exhibit a group of experimental results from the existing multivariate morphological filters, as shown in Figure 2, before researching multivariate dual morphological operators ( ⃗ , ⃗  denote multivariate opening and closing, respectively.The structuring element is a disk of size 3 × 3. Table 1 shows the popular vector orderings corresponding to filters).
As can be seen from Figure 2, multivariate opening and closing have different filtering performances.⃗  that started with erosion is able to effectively suppress noise, but ⃗  that started with dilation does not have the similar effect.⃗  is unable to effectively suppress noise and even enhance noise.Accordingly, we conclude that the existing multivariate morphological operators are not dual.To overcome the problem, multivariate dual morphological operators based on symmetric vector ordering (SVO) are proposed as below.
Based on the classical morphological combination operators and the definition of ( ⃗  SVO , ⃗  SVO ), multivariate opening and closing are denoted by ⃗  SVO and ⃗  SVO , respectively, We To demonstrate that ⃗  is able to maintain the brightness, saturation, and hue of color image, Figure 3 shows images processed by multivariate dual morphological operators and multivariate self-dual morphological operators (the structuring element is a square of size 3×3,    3 For multivariate self-dual morphological operators, As can be seen from Figure 3, images, shown in Figures 3(b1), 3(b2), and 3(b3), filtered by filtering operators that started with erosion will be darker than before.Similarly, images, seen in Figures 3(c1), 3(c2), and 3(c3), filtered by filtering operators that started with dilation will be brighter than before.However, Figures 3(d1), 3(d2), and 3(d3) show that images filtered by ⃗  maintain the gray value.In order to display them clearer, the histograms of Figure 3 in HSV color space are shown in Figure 4.

Multivariate Self-Dual Morphological Operators
As SDMO treats both bright and dark structures of an image in an identical way, it can suppress noise effectively and preserve the edges.However, for multichannel image filtering, VMF is more popular than SDMO since the former can achieve better effect than the latter.For different VMF, the popular method detects noise first and uses a switch to control filters.Consequently, the intelligent algorithm must be considered as a key step in multivariate SDMO in order to improve their performances.We analyze the basic theory of traditional morphological operators and soft morphological operators.Multivariate soft morphological erosion and dilation based on SVO are defined as [25,26] The th smallest value of the vector set The th largest value of the vector set , where  ⬦  = , .For the problem mentioned above, Evans and Liu proposed a novel pairwise pixel rejection scheme which works by removing the two pixels that are furthest apart and then finding the color morphological gradient of the remaining pixels.Based on the scheme, a morphological gradient approach to color edge detection is proposed to improve the performance in the presence of noise [27].In fact, this method employs extremum removal to reduce noise.In addition, Gimenez and Evans believed that the extrema definition used by color sieve is of critical importance for its subsequent performance [28].Inspired by these ideas, we proposed a novel algorithm of extremum computation based on extremum constraint.In the new algorithm, a constraint condition is used in extremum computation according to the principal of switched vector median filter.Figure 5(c) illustrates the principal of ⃗  EC SVO .And a more formal description for computing the extremum based on this procedure is given in Algorithm 1.
As to the supremum, it can be obtained in a likewise mode by simply sorting in an ascending order ≤ WED .Consequently, the resulting extrema can be used in order to define the operators in (9) for multivariate pixels.An illustration of this approach is given in Figure 5(c).And depending on Algorithm 1, ∧ EC SVO and ∨ EC SVO are defined as where (1) Sort in ascending order the vectors of k with respect to the ≤ BED ; Noise pixel According to ( 9) and ( 15), the basic multivariate morphological operators based on extremum constrain are defined as follows: where ⃗  EC BED and ⃗  EC WED are performed on Figure 6(b) and the results are shown in Figure 7.
Figure 7 shows that the optimized ⃗  EC SVO and ⃗  EC SVO output antinoise pixels.Therefore, we can conclude that MMO-EC can provide better antinoise filtering results than the existing multivariate morphological operators.According to the definitions of SDMO and ( ⃗  EC SVO , ⃗  EC SVO ), we can define multivariate self-dual morphological operators based on extremum constraint (MSDMO-EC) as follows: where  SVO is a multivariate morphological operators composed by ⃗  soft SVO and ⃗  soft SVO and  * SVO is the dual operation of  SVO .According to Definition 3, ⃗  EC SVO is also a multivariate self-dual morphological operator because

Experimental Results and Analysis
5.1.Noise Reduction.In this section, the proposed MSDMO-EC has been tested on more than 50 color images taken from public image databases.The image "Butterfly" is selected to show the capability of the proposed MSDMO-EC and is evaluated in detail to highlight the advantages of the algorithm.As mentioned above, VMF is one of the most popular approaches for impulsive noise removing, originally introduced by Astola et al. [29].The process consists of replacing the pixel value by the median of the pixels contained in a window around it.The existing VMF can provide better filtering results than various multivariate morphological filters.Consequently, for the evaluation of the efficiency of the proposed MSDMF-EC, the comparison with the following filtering operators was performed: (1) VMF: vector median filter (1990) [29] (2) AVMF: adaptive vector median filter (2003) [30] (3) FPGF: fast peer group filter (2005) [31] (4) LABSVMF: switching vector median filter in CIELAB color space (2007) [32] (5) RSVMF: robust switching vector median filter (2008) [33] (6) PSGF: peer group switching filter (2010) [34] (7) FRF: fuzzy rank-ordered differences filter (2010) [35] (8) QSF: quaternion switching filter (2012) [36].
Figures 8(a) and 8(a1) show the original image "Butterfly," and Figures 8(b) and 8(b1) show the original image contaminated by 25% salt and pepper noise.Figures 8(c)-8(j) and 8(c1)-8(j1) show images filtered by different VMF and the proposed MSDMO-EC (for comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3 and the structuring element of MSDMO-EC is also a square of size 3 × 3.  = ⃗  soft WED ,  * = ⃗  soft WED ,  = 2).Since VMF and the proposed MSDMO-EC are all selfdual operation, they can preserve the brightness, saturation, and hue of original image shown in Figures 8(c)-8(j) and 8(c1)-8(j1).Thus, the proposed MSDMO-EC overcomes the problem that the traditional vector morphological operators will lead to a heavily migration of the brightness, saturation, and hue of original image.
We can see that MSDMO-EC has the best performance in suppressing noise and preserving edges and structural features.In Figures 8(c)-8(i) and 8(c1)-8(i1), a lot of noise pixels are preserved or amplified while few noise pixels are preserved in Figures 8(j  MSDMO-EC is suitable and excellent for impulsive noise removing in multichannel image processing.
Figure 9 shows the dependence of the average SSIM, MSE, and NCD for 50 test images in the noise intensity ranging from 0.05% to 40%.The comparison with different filtering operators for noise reduction shows that the proposed MSDMF-EC can provide better results in the case of noise level over 10%.
Three quantitative measures: peak signal-to-noise ratio (PSNR), mean square error (MSE), and normalized color difference (NCD) are commonly used for the measurement of the restoration quality.Because structural similarity index measurement system (SSIM) provides a better measurement to image quality assessment, we use SSIM to replace PSNR [37,38] where  f and  g denote the average value (average brightness) of f and g, respectively. f and  g represent the variance of f and g.  fg is the covariance between f and g.  1 and  2 are the extremely minimum constant to avoid the case in which denominator is zero.
As can be seen from Figure 9, the various VMFs and MSDMF-EC are employed to restore image contaminated by different noise levels.Among these methods, the proposed MSDMF-EC provides higher SSIM and lower MSE and NCD.In addition, the experimental data show that the performance of different filtering operators is relatively close in the case of low noise levels.But with the increase of noise levels, the performance of various improved VMF declines quickly, even lower than the classical VMF.Obviously, the proposed MSDMF-EC is better than the traditional multivariate morphological filtering operators for maintaining brightness, hue, and saturation of images.Moreover, they have a significant advantage in handling salt and pepper noise pollution.

Color Image Segmentation.
Watershed transformation is an efficient tool for image segmentation.However, a direct computation of the watersheds of the segmentation function produces an oversegmentation which is due to the presence of spurious minima.Consequently, the image must be filtered before computing its watersheds so as to remove all irrelevant (1) image filtering using the filter; (2) calculating gradient using vector gradient approach proposed by di Zenzo [42]; (3) gradient reconstruction using morphological closingopening reconstruction operators [40]; (4) modifying gradient image using minimal imposition [43]; (5) watershed transformation.
According to the proposed segmentation approach and filtered images in Figure 8, we can compare the performance of various filters for color image segmentation.Figure 10 illustrates the segmentation results in terms of different approaches.It is obvious that the proposed MSDMF-EC is able to provide better preprocessing result for subsequent image segmentation (in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and ℎ = 0.03).
The proposed MSDMF-EC has a good performance for noise reduction and subsequent image segmentation.In order to further prove the performance of the proposed MSDMF-EC, we test the methods on natural image.Since natural images include various kinds of noise, it is necessary to apply the proposed morphological operators to natural images such as the standard Berkeley segmentation dataset.Figure 11 shows the segmentation results of natural images from BSD500.Comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3; the structuring element of opening-closing filter and MSDMO-EC is also a square of size 3 × 3 (for image segmentation, in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and ℎ = 0.03).
As can be seen from Figure 11, Figure 11(d) is the closest to Figure 11(a).In other words, the proposed MSDMO-EC can provide the best filtered images for watershed transformation.

Conclusions
In this study, we aim to find out why the existing multivariate morphological operators do not satisfy the duality.Motivated from this, we introduced a pair of symmetric vector orderings and applied it to construct multivariate dual morphological operators which can be used for multichannel image processing.Besides, we proposed MSDMO-EC based on the principle of extremum constraint to improve the performance of multivariate morphological operators for noise removal in multichannel images.Moreover, the advantages of the proposed operators are illustrated by two examples of color image filtering and segmentation.The experimental results demonstrate that the proposed MSDMO-EC is able to not only suppress noise efficiently but also provide better segmentation results.

Figure 1 :
Figure 1: Filtering results using morphological dual filtering operators and self-dual filtering operators.(a) Original image.(b) Image filtered by opening-closing.(c) Image filtered by closing-opening.(d) Image filtered by self-dual filtering operator .

Figure 6 :Figure 7 :
Figures 8(a) and 8(a1) show the original image "Butterfly," and Figures 8(b) and 8(b1) show the original image contaminated by 25% salt and pepper noise.Figures 8(c)-8(j) and 8(c1)-8(j1) show images filtered by different VMF and the proposed MSDMO-EC (for comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3 and the structuring element of MSDMO-EC is also a square of size 3 × 3.  = ⃗  soft WED ,  * = ⃗  soft WED ,  = 2).Since VMF and the proposed MSDMO-EC are all selfdual operation, they can preserve the brightness, saturation, and hue of original image shown in Figures8(c)-8(j) and 8(c1)-8(j1).Thus, the proposed MSDMO-EC overcomes the problem that the traditional vector morphological operators will lead to a heavily migration of the brightness, saturation, and hue of original image.We can see that MSDMO-EC has the best performance in suppressing noise and preserving edges and structural features.In Figures8(c)-8(i) and 8(c1)-8(i1), a lot of noise pixels are preserved or amplified while few noise pixels are preserved in Figures8(j) and 8(j1).Obviously, the proposed

Figure 11 :
Figure 11: Comparison of various morphological filters in terms of the segmentation results for natural image.(a) Ground-truth boundaries.(b) Segmentation results for images filtered by morphological opening-closing filter.(c) Segmentation results for images filtered by the VMF.(d) Segmentation results for images filtered by the MSDMO-EC.

Table 1 :
The popular vector orderings corresponding to filters.
. .,  ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟  is a structuring element. is called hard center;  \  is called soft boundary and  is the repetitive parameter.