^{1}

^{1}

^{2}

^{1}

^{2}

The earth mover's distance (EMD) is a measure of the distance between two distributions, and it has been widely used in multimedia information retrieval systems, in particular, in content-based image retrieval systems. When the EMD is applied to image problems based on color or texture, the EMD reflects the human perceptual similarities. However, its computations are too expensive to use in large-scale databases. In order to achieve efficient computation of the EMD during query processing, we have developed “fastEMD,” a library for high-speed feature-based similarity retrievals in large databases. This paper introduces techniques that are used in the implementation of the fastEMD and performs extensive experiments to demonstrate its efficiency.

The earth mover’s distance (EMD) was introduced in computer vision as an improved distance measure between two distributions, and it has been widely used in multimedia databases. In particular, in the area of content-based image retrieval (CBIR), where color images are retrieved from multimedia databases, it is important to apply improved distance measures. Several CBIR models have been proposed based on the histogram approach, such as the query by image content (QBIC) method proposed by Faloutsos et al. [

The similarity distance in the EMD is calculated as the amount of changes necessary to transform one image feature into another. Rubner and Tomasi [

The EMD is defined as a linear programming problem that can be solved using the simplex method. However, computation using the EMD and the simplex method is too complex for its application in interactive multimedia database systems. In order to achieve efficient retrieval processing in large-scale multimedia databases based on the EMD, we propose a fast retrieval algorithm for the EMD.

In order to reduce the computation times of the original distance, the proposed method uses the lower bounding distance described in Cohen and Guibas [

The remaining sections in this paper are organized as follows: Section

The

The Minkowski distance assumes different names depending on the value of

The

An illustration of perceptual dissimilarity.

Lp distance.

Perceptual dissimilarity.

Although the two histograms on the left are the same except for a shift by one bin, the

The quadratic-form distance is a representative of the cross-bin dissimilarity measures. Niblack et al. [

The EMD is based on the following linear programming problem. Let

We assume that the signature

Constraint (

An example of a two-dimensional EMD is shown. The signature

An example of the signature

Weight | |||
---|---|---|---|

0.3 | 5 | 5 | |

0.3 | 0 | 5 | |

0.3 | 0 | 0 | |

0.3 | 5 | 0 |

An example of the signature

Weight | |||
---|---|---|---|

0.4 | 1 | 1 | |

0.4 | 4 | 1 | |

0.4 | 2.5 | 4 |

Cost matrix of signature

5.6569 | 4.1231 | 2.6926 | |

4.1231 | 5.6569 | 2.6926 | |

1.4142 | 4.1231 | 4.7170 | |

4.1231 | 1.4142 | 4.7170 |

Optimized flow.

In Figure

Color distribution data applied for CBIR models is extracted from color segmentation images. As shown in Figure

Color signature for the EMD.

Sample of retrieved results using the EMD.

Sample of retrieved results using the

We denote a signature

The centroid

Suppose that

In the calculation of

An example is shown in which the signature

An example of the signature

Weight | |||
---|---|---|---|

0.5 | 5 | 5 | |

0.5 | 5 | 0 | |

0.2 | 2 | 7 |

Suppose that

The calculation cost of the

Flow in

0.4 | 0.4 | 0.4 | |

0.0000 | 0.0000 | 0.3000 | |

0.0000 | 0.0000 | 0.3000 | |

0.3000 | 0.0000 | 0.0000 | |

0.0000 | 0.3000 | 0.0000 |

From this characteristic, the flows can be calculated for each supply cluster. To calculate

An example of the calculation skip of the

In the case of the

Algorithm for the first

Algorithm from the

In a nearest-neighbor search based on a linear search,

There are various implementations of the priority queue, one of them being

Heap structure.

When a new search result offers a distance smaller than the largest value among the current results, the heap root (which has the largest value in the candidate list) is replaced with this newly found value. When this procedure causes a violation of the heap condition, the root value is exchanged with the largest of its children to restore the condition. If the heap condition is violated at the next level, a similar operation is applied; the procedure is terminated if a parent’s value is smaller than that of both children, or when the heap bottom is reached. This processing ensures that the data structure can be repaired so that every node meets the heap condition.

In order to evaluate this method, the number of color images in the database was increased from 5,000 to 50,000 (in increments of 5,000), and the retrieval time (CPU time) to search the top 10 images was evaluated. To evaluate the number of dimensions of the cluster representative

The experimental results are shown in Figures

CPU time on 5-dimensional data.

CPU time on 5-dimensional data (enlarged graph).

CPU time on 11-dimensional data.

CPU time on 11-dimensional data (enlarged graph).

From the experimental results, it can be seen that when the number of color images is 50,000 and has 5 dimensions, it takes approximately 2.5 seconds to retrieve the result for a query image by using the normal EMD calculation. The combination method of the lower bounds could obtain the same result as the normal EMD in approximately 0.15 second. On the other hand, the method using the calculation-skipping algorithm took only 0.1 second.

Some derivation algorithms of the EMD have been proposed. Pele and Werman [

The proposed method selects first

In order to validate that the hierarchical indexing tree can help to reduce the effects of outliers in the first _{top} and set to the initial threshold _{vp}, and set to the initial threshold

A preliminary experimental result.

5028.37 | 4723.46 | |

5115.23 | 2264.38 | |

4401.71 | 1969.05 | |

4071.22 | 2437.28 | |

4329.04 | 1588.82 | |

average | 4589.11 | 2596.60 |

In this paper, a fast retrieval method for the earth mover’s distance in multimedia databases is proposed. This method uses the lower bounding distance of EMD and combines it with a calculation-skipping algorithm. Moreover, the validity of the proposed method is supported by empirical observations. For future works, various multimedia information retrieval systems using the fast EMD should be implemented.

This work was supported in part by a grant from the Grant-in-Aid for Scientific Research nos. 17300036 and 21500940 from the Ministry of Education, Science, and Culture, Japan.