In view of the fact that the classical fast motion estimation methods are easy to fall into local optimum and suffer the high computational cost, the convergence of the motion estimation method based on the swarm intelligence algorithm is very slow. A new block motion estimation method based on single-dimension perturbation glowworm swarm optimization algorithm is proposed. Single-dimension perturbation is a local search strategy which can improve the ability of local optimization. The proposed method not only has overcome the defect of falling into local optimum easily by taking use of both the global search ability of glowworm swarm optimization algorithm and the local optimization ability of single-dimension perturbation but also has reduced the computation complexity by using motion vector predictor and terminating strategies in view of the characteristic of video images. The experimental results show that the performance of the proposed method is better than that of other motion estimation methods for most video sequences, specifically for those video sequences with violent motion, and the searching precision has been improved obviously. Although the computational complexity of the proposed method is slightly higher than that of the classical methods, it is still far lower than that of full search method.
Motion estimation (ME) is the prediction of the motion vector (MV) between the current frame and the reference frame. As an important part of video processing, it has been widely used in compression, computer vision, target tracking, and monitoring of industrial and other areas. Among many methods of ME, block matching algorithm (BMA) has been accepted by many video compression coding standards due to its simple structure and ease of implementation, for example, MPEG-1, MPED-4, H.261, H.263, and H.264. Full search (FS) [
Glowworm swarm optimization (GSO) [
In GSO, the agents are thought of as glowworms that carry a luminescent quantity called luciferin along with them. The glowworms encode the fitness of their current locations, evaluated using the objective function, into a luciferin value that they broadcast to their neighbors. The glowworm identifies its neighbors and computes its movements by exploiting an adaptive neighborhood, which is bounded above by a radial sensor range. Each glowworm selects, using a probabilistic mechanism, a neighbor that has a luciferin value higher than its own and moves toward it. These movements, based only on local information and selective neighbor interactions, enable the swarm of glowworms to partition into disjoint subgroups that converge on optima, but it also means that most glowworms will get together in the late iteration, which will lead to a slow optimum speed or even fall into local optimum. There is a typical example to illustrate the gathering process of GSO algorithm as showed in Figure
The gathering process diagram of basic GSO algorithm.
In this section, the basic glowworm swarm optimization algorithm and single-dimension perturbation strategy are introduced first, and then a novel glowworm swarm optimization algorithm based on single-dimension perturbation is proposed.
Here is the description of basic GSO algorithm.
In Luciferin-update phase, the luciferin updating of each glowworm depends on its current fitness value. That is to say, current luciferin value needs to add up a proportion of its current fitness value on the basis of its value at the previous moment. Besides, the luciferin value which is volatilized as the passage of time needs to be subtracted. The specific luciferin update formula can be stated as
In movement phase, neighbor glowworms which have higher luciferin value than its own are selected by each glowworm in its local-decision domain and form a neighborhood, and the moving object will be chosen from the neighborhood according to the probabilistic mechanism. For the glowworm
Movement equation is
In local-decision domain update rule, during the course of iteration, the number of neighbors is dynamic for each agent, so the local decision range must be adjusted to ensure that the swarm of glowworms can capture multiple peaks. The update rule is given by
Though GSO algorithm has many advantages such as strong universality, good overall and good performance at solving multi-objective optimization, it also has some shortcomings as well as other swarm intelligent algorithms, such as slow convergence speed and poor local search ability. According to these problems, a novel local search strategy called single-dimension perturbation search (SDPS) [
In 2D space, for instance, the four points which distance the glowworm
The computational procedure of the proposed algorithm can be summarized as follows.
Initialize the population. A swarm of
Update each glowworm Calculate the objective function value of the glowworm’s location— Select the neighbor glowworms in the local-decision domain— Calculate the probability— Select glowworm Let Calculate Update local-decision domain according to (
In iteration accumulation, if it reaches
Output the result.
The convergence of GSO algorithm is proved mainly through a large number of simulation experiments, and its mathematical theory basis is relatively weak. Krishnanand and Ghose analyzed the convergence of luciferin updating equation in 2006 [
Assuming that the luciferin update rule in (
For all glowworms
In order to show that the convergence of SDGSO algorithm has been improved compared against the standard GSO, and also better than differential evolution (DE) and ABC algorithm, three classical test functions are selected to do the convergence test. The formula of the functions, searching space, and theoretical optimal solution are given in Table
Formulas of the functions, searching space, and theoretical optimal solution.
Function | Formula | Searching space | Solution |
---|---|---|---|
Rastrigin |
|
|
0 |
Griewank |
|
|
0 |
Rosenbrock |
|
|
0 |
Comparison of DE, ABC, and GSO algorithm with SDGSO algorithm.
Function | Algorithm | Best | Median | Worst | Mean | Std |
---|---|---|---|---|---|---|
Rastrigin | DE |
|
|
|
|
26.70915 |
ABC | 79.52209 | 95.50064 |
|
95.38415 | 9.477102 | |
GSO |
|
0.001316 | 0.012433 | 0.002444 | 0.003363 | |
SDGSO |
|
|
|
|
0.000231 | |
| ||||||
Griewank | DE | 0.160226 | 0.270926 | 0.354931 | 0.266130 | 0.048583 |
ABC | 9.701455 | 24.90274 | 40.40466 | 25.53636 | 8.467937 | |
GSO |
|
0.001396 | 0.043757 | 0.007885 | 0.013473 | |
SDGSO |
|
|
|
|
0.000293 | |
| ||||||
Rosenbrock | DE |
|
|
|
|
42.24615 |
ABC |
|
|
|
|
|
|
GSO |
|
0.004393 | 0.222633 | 0.017098 | 0.048862 | |
SDGSO |
|
|
|
|
0.000232 |
It can be seen from Table
In order to illustrate the convergence speed of the improved algorithm more intuitively, convergence curves of GSO and SDGSO algorithms during the solving process of function Griewank and function Rosenbrock are shown in Figure
Convergence curves of GSO algorithm and SDGSO algorithm.
In order to statistically analyze the results in Table
SDGSO versus | DE | ABC | GSO |
---|---|---|---|
Rastrigin |
|
|
0.025639 |
Griewank |
|
|
|
Rosenbrock |
|
|
|
In block matching motion estimation, each frame image of video sequences is divided into macro block, and the blocks within the search window of the reference frame will be compared in a certain search pattern. Consequently the best matching block will be found, and the motion vector (MV) can be calculated as shown in Figure
Principle diagram of ME based on BMA.
GSO algorithm has strong commonality and a good performance of the low dimensional multimodal function optimization, especially suitable for block matching optimization of 2D images. In this paper, a novel block motion estimation method based on GSO algorithm which combines the good global searching characteristic of GSO algorithm with the single-dimension perturbation strategy and features of the video sequence is proposed. The proposed method not only has improved the accuracy of ME but also has decreased the average number of searching points. Specific implementation process is given as follows.
In BMA based on SDGSO algorithm, each macro block is regarded as glowworm, and the center location of the macro block is the location of glowworm. Initial population should be distributed within the search window of the reference frame after selecting the current macro block.
According to the statistics, video sequences have a characteristic called center biased distribution (CBD), and it means that most of the MVs will be concentrated in a central place. Besides, the MV of the current macro block has a similar size, and the same direction with its adjacent MVs, and the MV of the same block in the previous frame; that is to say, there is a strong space-time correlation among motion vectors. Therefore, we use the predictive MV to determine the search center. The average of the left block’s MV, the top block’s MV, the right-top block’s MV and the same block’s MV in the previous frame is regarded as the predictive MV, as shown in Figure
Motion vector predictor and the distribution of the initial population.
ME based on BMA is looking for the MV which can get the minimum (or maximum) value of the matching function. Matching function is an important factor of algorithm’s precision accuracy, and the mean average difference (MAD) is chosen as the matching function in this paper which has a small computation and convenient for hardware implementation, and it is defined as follows:
Luciferin-update equation is associated with matching function, defined as follows:
In the process of location updating, size of the step has a great effect on the convergence speed, especially for different motion intensity of video sequences. For example, when the movement is smooth like Akiyo sequence, a small step size like
Diagram of step size and search accuracy.
Aiming at this problem, the adaptive step size strategy [
After figuring out the step size, the location of the glowworm should be updated according to (
Due to the introduction of SDPS, the calculation of the objective function might be increased, but the speed of finding the optimized solutions is also accelerated. That is to say, the algorithm may find the optimal solution before it achieves the maximum iterations. Therefore, the following strategies are taken to terminate the iteration. Check whether the global optimal location has been changed within the prescribed continuous iterations and if it has not, the iteration should be terminated early. Check whether the maximum iteration is reached and if it is, terminate the iteration.
For more details, see Algorithm
In the experiments, 5 typical YUV video sequences are chosen according to the different motion intensity: Akiyo (very slow), Foreman (slow), Soccer (fast), Football (fast), and Bus (very fast). The first 100 frames of them are used as the test object, CIF format (352 × 188), 30 frames/second. Under the same test environment, full search (FS), three-step search (TSS), diamond search (DS), adaptive rood pattern search (ARPS), mutation particle swarm search optimization (MPSO) [
Parameters setting: size of macro block is 16 × 16, scope of search window is
In this paper, the peak signal-to-noise ratio (PSNR) is adopted as the evaluation criteria of search precision. The average PSNR value of the five test video sequences is listed in Table
Average PSNR of each algorithm.
Algorithm | PSNR (dB) | |||||
---|---|---|---|---|---|---|
Akiyo | Foreman | Soccer | Football | Bus | Average | |
FS | 39.98 | 32.22 | 28.76 | 24.93 | 24.32 | 30.04 |
DS | 39.98 | 31.94 | 27.64 | 23.54 | 21.46 | 28.91 |
TSS | 39.98 | 31.55 | 27.60 | 24.00 | 22.12 | 29.05 |
ARPS | 39.98 | 31.84 | 28.04 | 23.91 | 22.57 | 29.27 |
MPSO | 39.89 | 31.74 | 28.04 | 24.06 | 23.06 | 29.36 |
SDGSO | 39.91 | 31.79 | 28.27 | 24.10 | 23.32 | 29.48 |
It can be seen from Table
In order to show the good performance of SDGSO algorithm more intuitively, the restructuring images of the fourth frame of Soccer sequence are shown in Figure
The restructuring images of the fourth frame of Soccer by using different algorithms.
In order to prove that the application of GSO algorithm is better than that of ABC algorithm, the average PSNR value of the two algorithms which take the same population size, the same number of iterations, and the same prediction method of motion vector is listed in Table
Precision of ABC algorithm and GSO algorithm.
Algorithm | PSNR (dB) | |||||
---|---|---|---|---|---|---|
Akiyo | Foreman | Soccer | Football | Bus | Average | |
ABC | 38.38 | 31.39 | 27.96 | 23.42 | 23.07 | 28.84 |
GSO | 39.90 | 31.75 | 28.19 | 23.93 | 23.23 | 29.40 |
It can be seen from the results that the precision of ABC algorithm is worse than that of GSO algorithm no matter for video sequence with slow motion or that with fast motion. That is because the convergence speed of ABC algorithm is slower than that of GSO algorithm, and it cannot converge to the optimal value within the same iteration times. It is also worth noting that the precision of GSO algorithm in Table
Average search points of each algorithm.
Algorithm | Average search points (round off) | |||||
---|---|---|---|---|---|---|
Akiyo | Foreman | Soccer | Football | Bus | Average | |
FS | 869 | 869 | 869 | 869 | 869 | 869 |
DS | 11 | 15 | 25 | 34 | 20 | 21 |
TSS | 28 | 31 | 31 | 31 | 31 | 30 |
ARPS | 5 | 8 | 11 | 21 | 10 | 11 |
MPSO | 44 | 46 | 51 | 40 | 49 | 46 |
ABC | 38 | 42 | 43 | 47 | 42 | 42 |
GSO | 49 | 51 | 50 | 52 | 50 | 50 |
SDGSO | 30 | 34 | 33 | 43 | 35 | 35 |
In block matching motion estimation, the average number of search points for matching a block is used as the evaluation criterion of computation complexity. The average search point of each algorithm is listed in Table
It can be seen in Table
In order to prove that the introduction of SDPS has improved the performance, the application of SDGSO algorithm in ME is compared with that of the adaptive step glowworm swarm optimization (AGSO) algorithm from two aspects of search precision and computation complexity as shown in Tables
Average PSNR of AGSO and SDGSO.
Akiyo | Foreman | Soccer | Football | Bus | |
---|---|---|---|---|---|
AGSO | 39.68 | 31.25 | 27.94 | 22.89 | 23.17 |
SDGSO | 39.71 | 31.79 | 28.27 | 24.10 | 23.32 |
Average search points of AGSO and SDGSO.
Akiyo | Foreman | Soccer | Football | Bus | |
---|---|---|---|---|---|
AGSO | 31.10 | 30.24 | 33.28 | 36.91 | 36.66 |
SDGSO | 30.00 | 34.20 | 33.31 | 43.21 | 34.90 |
It can be seen from Tables
In this paper, a novel glowworm swarm optimization algorithm based on single-dimension perturbation search strategy is proposed and applied to block motion estimation (Algorithm
(3.1) (3.2) (3.3) (3.4) Let (3.5)
block.
(4.5.1) for each glowworm (4.5.1.1) Form the neighborhood according (4.5.1.3) Select glowworm (4.5.1.4) Update glowworm step (4.5.1.5) Update glowworm position (4.5.1.6) Search the new location by using SDPS; (4.5.1.7) Update the luciferin value (4.5.1.8) Update local-decision domain (4.5.1.9) if iteration;
This research is supported by the National Science Foundation Council of Guangxi (2012GX NSFAA053227).