In traditional adaptive-weight stereo matching, the rectangular shaped support region requires excess memory consumption and time. We propose a novel line-based stereo matching algorithm for obtaining a more accurate disparity map with low computation complexity. This algorithm can be divided into two steps: disparity map initialization and disparity map refinement. In the initialization step, a new adaptive-weight model based on the linear support region is put forward for cost aggregation. In this model, the neural network is used to evaluate the spatial proximity, and the mean-shift segmentation method is used to improve the accuracy of color similarity; the Birchfield pixel dissimilarity function and the census transform are adopted to establish the dissimilarity measurement function. Then the initial disparity map is obtained by loopy belief propagation. In the refinement step, the disparity map is optimized by iterative left-right consistency checking method and segmentation voting method. The parameter values involved in this algorithm are determined with many simulation experiments to further improve the matching effect. Simulation results indicate that this new matching method performs well on standard stereo benchmarks and running time of our algorithm is remarkably lower than that of algorithm with rectangle-shaped support region.
Stereo vision is a fundamental technique for extracting 3D information of a scene from two or more 2D images. It is widely applied in robot navigation, remote sensing, and industrial automation. One of the key technologies of stereo vision is stereo matching, which produces a disparity map. The stereo matching algorithm can be classified into two broad categories: global-based and local-based algorithms.
Global-based matching algorithms follow the energy minimization principle. First, an energy function is established, consisting of a data term and a smoothness term. Next, this function is minimized with a global optimization method. Dynamic programming [
A local-based matching algorithm is a simple and effective method for stereo matching that is commonly used. An important underlying principle of local-based matching is that pixels in a support region have an approximately equal disparity. To satisfy this principle, it is very important to determine the support region size. The support region must contain enough pixels for intensity variation, and the support region must include only those pixels with the same disparity. Thus, the traditional, local-based matching method is prone to false matching for pixels from the depth discontinuities region, since those pixels are from different depths. To ensure that a local-based matching algorithm performs well in practical applications, various approaches have been proposed. For example, adaptive windows have been used to improve matching results. These methods search an appropriate support region for each pixel, greatly improve the performance of matching results, and outperform standard local-based methods [
In recent years, several methods for acquiring satisfactory effect of stereo matching have been adopted. Yang et al. [
The algorithm presented in this paper is inspired from adaptive-weight matching algorithm. In this paper, the aim is to propose a low computation complexity and high accuracy stereo matching algorithm. So the rectangle-shaped support region is substituted for the line-shaped support region. Lacking of enough pixel information is a main weakness of the line-shaped support region, which is easy to cause error matching. Adaptive-weight can make full of limited pixel information, by analyzing the characteristic of the adaptive-weight model proposed in [
In addition, several approaches are applied to complete the algorithm. We develop a new pixel dissimilarity measurement function which combines Birchfield pixel dissimilarity measurement function and census transform to compute the matching cost. The loopy belief propagation method proposed in [
The algorithm presented in this paper can be divided into two steps: a disparity map initialization step and a refinement step. The framework of the algorithm is shown in Figure
Block diagram of our algorithm.
Assuming the two pixels
Wang et al. [
The assumption of segmentation-based stereo matching.
Figure
Spatial proximity described by neural network.
Spatial proximity proposed in this paper when
In Figure
According to the segmentation-based stereo matching principle [
Equation (
That color similarity model based on image segmentation can achieve good performance, as introduced in [
The matching cost of pixel
The pixel dissimilarity measurement function
To validate the effect of this cost aggregation method, simulation results on Teddy and Cones with Birchfield method and our method are shown in Figure
(a) Part of the origin image. (b) Disparity results which are computed with our cost aggregation method. (c) Disparity results which are computed with Birchfield pixel dissimilarity measurement function.
The winner-take-all (WTA) searching strategy is a common method for determining disparity, which can be expressed by
WTA tends to produce a low accuracy disparity map. Therefore, we adopt an efficient LBP algorithm proposed in [
( Initializing pyramid level ( Pyramid( ( ( ( ( ( ( (
It is inevitable that initial disparity maps will contain many error-matched pixels. To refine the disparity map, a two-step postprocessing method is put forward in this section.
The disparity map
In this work, we analyzed the property of initial disparity map. Figure
The distribution of bad pixels for initial disparity map of Tsukuba. Bad pixels are marked with red points.
Pixels can be divided into two types: undependable pixels and dependable pixels. Pixel
Pixel
( ( ( ( ( ( ( ( ( ( ( (
Figure
The error percentages in different region for disparity refinement step.
Nonocc | All | Disc | |
---|---|---|---|
Initial disparity map | 1.21% | 3.20% | 6.42% |
Left-right consistency | 1.17% | 2.40% | 6.30% |
Segmentation voting | 0.95% | 1.62% | 5.07% |
The disparity map after execution of left-right consistency checking.
The pixelwise region shown in Figure
Pixelwise region for segmentation voting.
Let
( ( ( ( ( ( ( (
The
The disparity map of Tsukuba obtained by segmentation voting.
The parameters involved in our algorithm greatly affect the performance of the algorithm. In this section, we present the parameter settings.
We considered eight main parameters:
Figure
(a) Performance evaluation of the proposed method when varying
Figure
(a) Performance evaluation of the proposed method according to
Figure
(a) Performance evaluation of the proposed method when varying
Figure
(a) Performance evaluation of the proposed method according to
For the disparity map refinement step, two main parameters must be set.
(a) Performance evaluation of the proposed method according to
We evaluate our algorithm on Middlebury benchmarks [
Parameter setting for Middlebury benchmark.
|
|
|
|
|
|
|
|
---|---|---|---|---|---|---|---|
40 | 1 | 15 | 8 | 15 | 5 | 7 | 12 |
Simulation results on Middlebury data sets are presented in Figure
Quantitative evaluation results for Middlebury benchmark.
Algorithm | Tsukuba | Venus | Teddy | Cones | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Nonocc | All | Disc | Nonocc | All | Disc | Nonocc | All | Disc | Nonocc | All | Disc | |
AdaptingBP | 1.1122 | 1.379 | 5.7924 | 0.104 | 0.217 | 1.448 | 4.2217 | 7.0615 | 11.818 | 2.4822 | 7.9228 | 7.3227 |
CoopRegion [ |
0.875 | 1.162 | 4.614 | 0.115 | 0.215 | 1.5414 | 5.1630 | 8.3120 | 13.025 | 2.7939 | 7.1814 | 8.0147 |
Our algorithm | 0.898 | 1.5923 | 4.789 | 0.115 | 0.3624 | 1.4911 | 4.3318 | 9.9332 | 11.217 | 2.8141 | 8.4645 | 7.8540 |
RDP [ |
0.9711 | 1.3911 | 5.0011 | 0.2139 | 0.3829 | 1.8924 | 4.8422 | 9.9433 | 12.622 | 2.5325 | 7.6920 | 7.3828 |
MultiRBF [ |
1.3347 | 1.5620 | 6.0232 | 0.1311 | 0.172 | 1.8421 | 5.0928 | 6.368 | 13.429 | 2.9048 | 6.768 | 7.1024 |
The results of whole performance evaluation for Middlebury benchmark.
Algorithm | Average rank | Average percent bad pixels |
---|---|---|
AdaptingBP | 16.5 | 4.23 |
CoopRegion | 12.2 | 4.41 |
Our algorithm | 22.8 | 4.48 |
RDP | 22.9 | 4.57 |
MultiRBF | 23.2 | 4.39 |
(a) The final disparity map obtained by our algorithm. (b) Error maps. (c) Ground truth.
These results demonstrate our algorithm has a good performance. However, it is difficult to decrease the error percentages in the three regions (nonocc, all, and disc) at the same time. This is because many pixels with correct disparity are classified as undependable, according to (
Erroneous disparity distribution determined by (
Figure
Comparison between standard LRC checking and iterative LRC checking presented in Section
Algorithm | Nonocc | All | Disc |
---|---|---|---|
Initial disparity | 1.21% | 3.20% | 6.42% |
Standard LRC checking | 1.43% | 2.10% | 7.54% |
Iterative LRC checking | 1.17% | 2.40% | 6.30% |
Erroneous disparity distribution after executing standard LRC checking for Tsukuba.
Thus, it can be seen that the error percentage in the disc region greatly increases after applying standard LRC checking. Because the disc region is part of the nonocc region, the error percentage in the nonocc region also increases. Our iterative LRC checking method can effectively improve the performance of our stereo matching algorithm.
In this section, we investigate the computation running time of our algorithm. The running time directly reflects computational complexity. Without a loss of generality, our algorithm runs 50 times and average running time was calculated. Results are shown in Table
The average running time of each part of our algorithm for Tsukuba.
Mean-shift | Cost aggregation | LBP | Iterative LRC | Segmentation voting |
---|---|---|---|---|
7.37 | 19.23 | 0.78 | 0.24 | 0.30 |
The new rectangle-based adaptive-weight method can be obtained by imposing the
The variation of average running time of cost aggregation of our rectangle-based algorithm under different size of support region.
The size of support region |
|
|
|
|
|
---|---|---|---|---|---|
Running time | 37.02 | 69.81 | 106.53 | 154.41 | 213.94 |
Our algorithm with the rectangle-shaped support region performed best when the size of the support region is
In this work, we proposed a new line-based adaptive-weight stereo matching algorithm that integrates several methods. The main conclusions that can be drawn from our results are as follows. Cost aggregation is the most time-consuming part of stereo matching algorithm. Using a line-shaped support region can dramatically reduce the elapsed time of cost aggregation. The adaptive-weight model proposed in this paper can produce a rather satisfactory initial disparity map in the absence of enough pixel information. Experimental results show that the algorithm proposed in this paper can attain a better matching effect with less running time.
Although our algorithm has a good performance on Middlebury data sets, there is still much room for improvement. There are too many parameters in our algorithm to accommodate different image pairs. In further research, we will analyze the intrinsic relationship among the parameters and reduce the number of parameters. Figure
The half-length of support region
Shape controlling parameter
The half-length of transition region
Spatial radius of mean-shift
Color radius of mean-shift
The half-length of segment for segmentation voting
Confidence level of color similarity for segmentation voting.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors express their appreciation for the financial support of the Shandong Natural Science Foundation, Grant no. ZR2013FL033. They also extend their sincere gratitude to reviewers for their constructive suggestions.