Otsu’s algorithm is one of the most well-known methods for automatic image thresholding. 2D Otsu’s method is more robust compared to 1D Otsu’s method. However, it still has limitations on salt-and-pepper noise corrupted images and uneven illumination images. To alleviate these limitations and improve the overall performance, here we propose an improved 2D Otsu’s algorithm to increase the robustness to salt-and-pepper noise together with an adaptive energy based image partition technology for uneven illumination image segmentation. Based on the partition method, two schemes for automatic thresholding are adopted to find the best segmentation result. Experiments are conducted on both synthetic and real world uneven illumination images as well as real world regular illumination cell images. Original 2D Otsu’s method, MAOTSU_2D, and two latest 1D Otsu’s methods (Cao’s method and DVE) are included for comparisons. Both qualitative and quantitative evaluations are introduced to verify the effectiveness of the proposed method. Results show that the proposed method is more robust to salt-and-pepper noise and acquires better segmentation results on uneven illumination images in general without compromising its performance on regular illumination images. For a test group of seven real world uneven illumination images, the proposed method could lower the ME value by 15% and increase the DSC value by 10%.
As a fundamental technique for computer vision related applications, image segmentation has been studied for decades [
To improve the performance of Otsu’s method, many scholars have done significant amount of work to study its characteristics and improve the thresholding algorithm since it was proposed. Many one-dimensional (1D) improved Otsu’s algorithms are firstly developed. Xu et al. studied the characteristics of the threshold value acquired using Otsu’s method and came to the conclusion that the threshold biases toward the class with larger variance when the intraclass variances of two classes are different [
In order to overcome the disadvantage shared by most 1D Otsu’s algorithms, Liu et al. proposed to extend Otsu’s method to a 2D histogram. The 2D Otsu’s method utilizes both the pixel’s gray level and the average gray level of its neighborhood and is experimentally proved to perform better than 1D Otsu’s method on images corrupted by noise [
Due to the stability of environment light, the object size, and other reasons, uneven illumination is a common problem in image capturing [
As introduced above, the existing uneven methods usually not directly improve the original segmentation algorithm. For better overcoming the disadvantages of 2D Otsu’s algorithm, in this paper, we are focusing on developing an improved 2D Otsu’s method to generate better segmentation performance on both uneven illumination images and salt-and-pepper corrupted images. The main contributions of this paper are as follows: (1) firstly, a novel 2D histogram construction strategy based on median and average filters is proposed to enhance the algorithm’s robustness to salt-and-pepper noise. (2) Secondly, an energy based partitioning technology is introduced to find the best splitting line for uneven illumination images. (3) Lastly, two schemes based on the new 2D histogram construction strategy and partitioning method are proposed, and their robustness to salt-and-pepper noise is studied and the segmentation performance is evaluated both qualitatively and quantitatively on uneven illumination testing images and on an extra cell dataset. The remainder of this paper is organized as follows. In Section
In this section we will briefly review the traditional 2D Otsu’s method firstly. Suppose
Figure
Image rice and its histogram, 2D histogram, and corresponding division. (a) Original image. (b) Histogram. (c) Projection of 2D histogram. (d) Corresponding regional division of 2D histogram.
The corresponding mean vectors of
Due to the assumption that occurrence of image data away from the diagonal of 2D histogram is negligible, we can get the following approximate expression:
The between-class variance in 2D Otsu’s algorithm can be then defined as
Then, we can get the optimal threshold pair
While 2D Otsu’s algorithm is more robust to noises than 1D Otsu’s method, it still has limitations on salt-and-pepper noise corrupted images as well as uneven illumination images. Figure
Segmentation results on salt-and-pepper corrupted image. (a) Image coins corrupted by salt-and-pepper noise (
In addition to salt-and-pepper noise, traditional 2D Otsu’s method usually produces poor segmentation on uneven illumination images. Figure
Segmentation results on an uneven illumination image. (a) Image rice. (b) Segmentation result of traditional 2D Otsu’s method.
To solve the abovementioned problems of 2D Otsu’s algorithm, in this paper, we focus on 2D histogram constructing to enhance the robustness of 2D Otsu’s method to salt-and-pepper noise, and the image partitioning technology is studied to improve the algorithm’s effectiveness on uneven illumination images.
The workflow of our proposed method is shown in Figure
Block diagram of the proposed robust Otsu’s algorithm.
As discussed in Section
Firstly, we apply median filtering with
After we get the median image
Finally, we construct the 2D histogram using median image
Figure
Projection of 2D histograms on image coins corrupted by salt-and-pepper noise (
Due to the limitation of traditional 2D Otsu’s method for uneven illumination images, in this section we explore partition strategy to improve the algorithm’s segmentation ability. It is a very straightforward idea to divide the uneven illumination images into some patches in which the illumination is uniform. However, how to divide the uneven illumination image adaptively is the key problem to be solved. Seam-line technology is widely used in image stitching [
In order to compute the color energy for the first row, we add image padding using the first row of the image in our implementation.
The other term
In addition to the above discussed requirements for the splitting line, we add position weight to the energy function to make the divided two parts as even as possible. The position weight is defined using a Gaussian function defined below:
For an image of size
The energy function is finally changed to formula (
With the above given energy function, we define a splitting line as an optimal one if it gets the maximum average energy. In order to search for the optimal splitting line, we use dynamic programming associated with formula ( Step 1: initialization: as shown in Figure Step 2: expanding sequentially until reaching the last column: for a pixel with Step 3: find the maximum cumulative energy in pixels of the last column and backtrack the optimal split line using matrix An uneven illumination image can be divided into two parts whose illumination is more consistent using the above introduced optimal splitting line search method. Figure
Illustration of the optimal splitting line search algorithm. Each gray filled circle represents its maximum cumulative energy value. (a) Initialization. (b) Expanding for the first time.
Optimal splitting lines of some uneven illumination images.
After implementing the partition step, we can split the processed image I into two parts denoted as Ip1 and Ip2. In this section, we will discuss automatic thresholding segmentation for uneven illumination images based on the two split parts, Ip1 and Ip2. There are two typical strategies, which are considering Ip1 and Ip2 separately or computing a uniform threshold for both Ip1 and Ip2. Consequently, we propose two schemes for uneven illumination image thresholding based on the partition technology. For scheme one, the improved 2D Otsu’s method MAOTSU_2D is directly implemented on Ip1 and Ip2 separately, and we can obtain two threshold vectors for the corresponding two parts. On the other hand, we proposed a second scheme to calculate a uniform threshold vector. The main steps of the proposed second scheme are as follows: Step 1: generate 2D histogram using the improved constructing strategy proposed in Section Step 2: calculate the between-class variance Step 3: compute the optimal threshold pair The object function of the proposed scheme 2 includes two terms,
In order to verify the effectiveness of the proposed methods, we conducted experiments on a personal computer with Intel Core 2.30 GHz CPU and 4.0 GB memory. Original 2D Otsu’s method, MAOTSU_2D without postprocessing, and MAOTSU_2D, as well as latest two 1D Otsu’s methods (Cao’s method and DVE) were selected to compare with our proposed schemes. All algorithms are implemented using Matlab R2012b, and experiments are conducted for verifying algorithms’ robustness to salt-and-pepper noise (Section
For quantitative testing, misclassification error (ME) and dice similarity coefficient (DSC) are adopted as the evaluation metrics. The definition of ME is described as follows:
The other evaluation metric DSC can be expressed as the following formula:
In this section, we experimentally discuss the robustness of the proposed methods to salt-and-pepper noise. The testing image is the widely used image coins, and we had manually labelled the binary segmentation result as the ground truth as shown in Figure
Testing image coins and corresponding ground truth. (a) Image coins. (b) Manually labelled ground truth.
Figure
Relationship between ME and value of noise intensity parameter
Figure
Segmentation results of each algorithm on salt-and-pepper corrupted images.
In this section, experiments are conducted to verify the effectiveness of our proposed schemes on uneven illumination images. We test different methods on both synthetic and real world images as shown in Figure
Synthetic and real world testing images and their corresponding ground truth. (a) Original images. (b) Corresponding manually labelled ground truth.
Table
ME and DSC values of each algorithm on synthetic and real world images.
ME and DSC | ||
---|---|---|
Synthetic image | Real world image (rice) | |
Original 2D Otsu | 0.2806, 0.7097 | 0.0524, 0.9074 |
MAOTSU_2D | 0.2774, 0.7121 | 0.0527, 0.9077 |
Cao’s | 0.1397, 0.7839 | 0.0686, 0.8750 |
DVE | 0.4359, 0.6425 | |
Proposed scheme 1 | 0.0425, 0.9267 | |
Proposed scheme 2 | 0.1232, 0.8154 | 0.0512, 0.9141 |
Bold values represent the best evaluation metric values among all the compared algorithms.
Segmentation results of each algorithm on the synthetic and real world images.
Figure
For further verification, we test our method on some other real world uneven illumination images shown in Figure
Segmentation results of each algorithm on other uneven illumination images.
Next, we evaluate the effectiveness of all testing methods quantitatively. Tables
ME values of each algorithm on real world uneven illumination images.
Image | Original 2D Otsu | MAOTSU_2D | Cao’s | DVE | Proposed scheme 1 | Proposed scheme 2 |
---|---|---|---|---|---|---|
#1 | 0.0404 | 0.0418 | 0.0413 | 0.2945 | 0.0298 | |
#2 | 0.2056 | 0.2058 | 0.0596 | 0.0303 | 0.0370 | |
#3 | 0.3512 | 0.3618 | 0.3659 | 0.3274 | 0.2745 | |
#4 | 0.3995 | 0.4053 | 0.4137 | 0.3336 | 0.1790 | |
#5 | 0.0808 | 0.0798 | 0.0893 | 0.1125 | 0.0774 | |
#6 | 0.0349 | 0.0339 | 0.0352 | 0.0380 | 0.0336 | |
#7 | 0.6754 | 0.6808 | 0.7771 | 0.5149 | 0.5970 | |
Avg. | 0.2554 | 0.2585 | 0.2546 | 0.2337 | 0.1598 |
Bold values represent the best ME values among all the compared algorithms.
DSC values of each algorithm on real world uneven illumination images.
Image | Original 2D Otsu | MAOTSU_2D | Cao’s | DVE | Proposed scheme 1 | Proposed scheme 2 |
---|---|---|---|---|---|---|
#1 | 0.9755 | 0.9747 | 0.9773 | 0.8032 | 0.9817 | |
#2 | 0.8710 | 0.8709 | 0.9645 | 0.9815 | 0.9796 | |
#3 | 0.6857 | 0.6732 | 0.6682 | 0.7132 | 0.7701 | |
#4 | 0.6753 | 0.6688 | 0.6604 | 0.774 | 0.8789 | |
#5 | 0.9170 | 0.9180 | 0.9074 | 0.8803 | 0.9207 | |
#6 | 0.7306 | 0.7338 | 0.7265 | 0.7100 | 0.7361 | |
#7 | 0.3207 | 0.3162 | 0.2964 | 0.3741 | 0.3408 | |
Avg. | 0.7394 | 0.7365 | 0.7430 | 0.7543 | 0.8090 |
Bold values represent the best DSC values among all the compared algorithms.
In order to evaluate the performance of the proposed methods on other real world images, in this section, all the compared algorithms are tested on a cell dataset, which was introduced by Xing et al. in [
Figure
Segmentation results of each algorithm on cell images.
The average ME and DSC values of each algorithm on all 22 cell images can be found in Table
Average ME and DSC values of each algorithm on cell images.
Metric | Original 2D Otsu | MAOTSU_2D | Cao’s | DVE | Proposed scheme 1 | Proposed scheme 2 |
---|---|---|---|---|---|---|
ME | 0.1606 | 0.1492 | 0.1639 | 0.1457 | 0.1374 | |
DSC | 0.7302 | 0.7709 | 0.7148 | 0.7831 | 0.7803 |
Bold values represent the best ME and DSC values on cell images among all the compared algorithms.
In the previous experimental sections, robustness and segmentation performance of the improved 2D Otsu’s method are tested. The improved method can deal with salt-and-pepper noise properly compared to original 2D and 1D Otsu’s methods and can make a significant improvement on segmenting uneven illumination images both qualitatively and quantitatively. At the same time, for normal cell images segmentation, the performance of the proposed method is still quite competitive. However, as a global thresholding method, there exists a performance limit, which means that, for some uneven images, no global threshold could be found that can effectively extract foreground. Moreover, the foreground detection ability of the proposed method on the cell dataset is slightly weaker than our previous DVE method which can be found partially in Figure
To alleviate the limitations of 2D Otsu’s method to salt-and-pepper noise and uneven illumination for image segmentation, we have proposed a robust 2D Otsu’s method in this paper. A 2D histogram construction strategy based on median and average filters is introduced, and an energy based image partition method is developed for uneven illumination image segmentation. Experiments are conducted on both synthetic and real world images to evaluate the performance of the proposed method. The qualitative and quantitative evaluations collectively indicate that the proposed method is more robust to salt-and-pepper noise and performs better on uneven illumination images compared with original Otsu’s method, MAOTSU_2D, Cao’s method, and DVE, demonstrating its better flexibility in automatic image thresholding.
The data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that they have no conflicts of interest.
This work was partly supported by the National Natural Science Foundation of China (Grant nos. 61866031, 61862053, 61762074, and 31860030) and the Science Technology Foundation for Middle-aged and Young Scientist of Qinghai University (Grant nos. 2016-QGY-5, 2017-QGY-4, and 2018-QGY-6).