JAM Journal of Applied Mathematics 1687-0042 1110-757X Hindawi Publishing Corporation 10.1155/2014/592834 592834 Research Article Optimization of the Regularization in Background and Foreground Modeling Wang Si-Qi Feng Xiang-Chu Zhang Zhihua 1 School of Mathematics and Statistics Xidian University Xi’an 710126 China xidian.edu.cn 2014 1182014 2014 17 05 2014 16 07 2014 16 07 2014 11 8 2014 2014 Copyright © 2014 Si-Qi Wang and Xiang-Chu Feng. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background and foreground modeling is a typical method in the application of computer vision. The current general “low-rank + sparse” model decomposes the frames from the video sequences into low-rank background and sparse foreground. But the sparse assumption in such a model may not conform with the reality, and the model cannot directly reflect the correlation between the background and foreground either. Thus, we present a novel model to solve this problem by decomposing the arranged data matrix D into low-rank background L and moving foreground M . Here, we only need to give the priori assumption of the background to be low-rank and let the foreground be separated from the background as much as possible. Based on this division, we use a pair of dual norms, nuclear norm and spectral norm, to regularize the foreground and background, respectively. Furthermore, we use a reweighted function instead of the normal norm so as to get a better and faster approximation model. Detailed explanation based on linear algebra about our two models will be presented in this paper. By the observation of the experimental results, we can see that our model can get better background modeling, and even simplified versions of our algorithms perform better than mainstream techniques IALM and GoDec.

1. Introduction

Over the past several decades, the applications of monitoring and control system, including video surveillance, traffic monitoring and analysis, human detection and tracking, and gesture recognition in human-machine interface, are increasingly popular and widely used in our daily life. Moeslund et al.  and many scholars were committed to the related researches by using various methods.

Background subtraction is an algorithm proposed since 1999. Due to the underlying ease of implementation and effectiveness, foreground object detection by using background subtraction has been widely used in the video surveillance applications. This kind of approach can be traced back to Toyama et al. , the algorithm which predicts pixel intensity by using the Wiener filter to dig out the foreground region instead of maintaining a specific image. In the meanwhile, Stauffer and Grimson [3, 4] proposed a method based on the Gaussian mixture model (GMM) to track the dynamic objects and give the trajectory of them. The GMM has gained great popularity among the computer vision community, and other scholars continue to revisit the method and propose enhanced algorithms [5, 6]. But modeling the high frequency variations in the background with 3–5 of Gaussian distribution is not accurate and even fails to achieve the sensitive detection. So, in the following studies, Elgammal et al. [7, 8] proposed a corresponding new background subtraction, using more short-term distributions to obtain better detection sensitivity. They put forward the statistical scene background by using general nonparametric kernel density estimation to subtract the background while retaining the foreground image. All the methods above are real-time algorithm; the estimated background at each pixel location is based on the pixels recent history and no spatial correlation is used between different or neighbouring pixel locations. So, Oliver built an eigenspace to model the background and used principal component analysis (PCA) to reduce the dimensionality of the space .

Over the last five years, there has been growing interest within the usage of PCA to model the background. Chandrasekaran put forward the “low-rank + sparse” model to achieve the purpose of background modeling and foreground detection in the video processing field . In 2009, Candes et al.  proposed the robust principal component analysis (RPCA) method to decompose the large data matrix of video sequence into low-rank and sparse components. RPCA being used firstly in the field of computer vision was put forward by de la Torre and Black . Almost at the same time as Candes, Wright et al.  also chose RPCA to model the background as a low-rank part of the frames in the video sequence. After that, many scholars threw themselves into the work to solve this model better. An alternating direction method (ADM) for sparsity and low-rank decomposition was studied by Yuan and Yang . Lin et al. solved this optimization problem via the method of augmented Lagrange multipliers (ALM) in . Zhou and Tao  introduced a special fast low-rank approximation based on bilateral random projections (BRP), which was the essence of their algorithm GoDec. Peng et al.  referred to RPCA problem as robust alignment by sparse and low-rank (RASL) decomposition and gave the outer and inner loop of RASL to solve the problem. All these improved PCA methods exploit the correlations between different pixels very well and no longer being local method like the previous background subtraction algorithms.

The general “low-rank + sparse” model in the articles available is as follows: (1) min L , S 1 2 D - L - S F 2 + λ L * + μ S 1 , where D is a data matrix whose columns are the pixels from each frame in the video sequence; L denotes a low-rank matrix which naturally corresponds to the stationary background; and S is a sparse matrix which contains the moving objects in the foreground. The Frobenius norm · F 2 is defined as the quadratic sum of the matrix elements; the nuclear norm · * is defined as the sum of its singular values; the l 1 -norm · 1 is seen as the sum of the absolute values of the matrix elements [18, 19].

To address this convex optimization problem, numerous approaches have been explored and proposed in the academic papers over several decades. However, in my opinion, the “sparse” assumption in this model may not always conform to the reality. The foreground moving target may be a very small part of the full panoramic view, and it may also occupy a large proportion in the captured picture. And, under normal circumstances, the scholars usually add a related item onto (1) to constraint the correlation between foreground and background matrices. Therefore, we are trying to improve the PCA method in another way to find a new model to separate the background and foreground and reflect the correlation between them as well. In this paper, we proposed a “low-rank + dual” model to solve this foreground and background modeling problem. And we further use the reweighted dual function norm instead of the normal norms so as to get a better and faster model, as shown in Algorithm 2. Although our two models are all based on offline method, they provide the probability distribution function of the background within the consideration of the spatial correlation between neighboring pixels. Experiments on several scenes show that our methods have better performance and lower consumption compared with the “low-rank + sparse” methods, IALM and GoDec.

This paper is organized as follows. Section 2 briefly details our motivation of proposing the rudimentary dual model, and the algorithm to this dual model will also be given; a further improved reweighted form model and the corresponding algorithm will be given in Section 3; and, in Section 4, the experiments of our methods will be presented and the results will be compared with IALM and GoDec; and a final summary will be given in Section 5.

2. Dual Norm Model 2.1. Functional with the Dual Norm Constraint

Inspired by Meyer for the usage of the dual norms to regular the cartoon and texture in the image decomposition problem , we try to use the advantages of such dual regularizations. So, we put forward our rudimentary dual norm model, “low-rank + dual” model, which has been improved from the general “low-rank + sparse” model, as shown in Algorithm 1. Here, we keep the low-rank hypothesis of the model for the background, which is similar to Oliver’s concept of eigenspace, and use the nuclear norm as one regularization to realize this assumption. But, for the foreground part, we use the “dual” norm regularization instead of the “sparse” constraint to obtain the most uncorrelated foreground from the background. To illustrate this point, we need to recall the definition of the dual norm .

<bold>Algorithm 1: </bold>Solving dual norm model NSMP with AIM.

(1) Input: D ,

(2) initialize: L 0 = D - , k = 0 , λ > 0 , μ > 0 .

(3) repeat

(4)  k = k + 1 ;

(5)  M k = T τ ( D - L k - 1 ) ; [ calculate τ with (17) ]

(6)  L k = S λ ( D - M k ) .

(7) until a stopping criterion is satisfied.

(8) output: L * , M * .

<bold>Algorithm 2: </bold>Solving WNSMP with reweighted dual function.

(1) Input: D ,

(2) initialize: M 0 = 0 , k = 0 , λ > 0 , μ > 0 , c > 0 , δ > 0 .

(3) repeat

(4)  k = k + 1 ;

(5)  w i k c n / ( Σ i ( D - M k - 1 ) + δ ) ;

(6)  L k = T λ w ( D - M k - 1 ) ;

(7)  M k = T τ / w ( D - L k ) . [ calculate τ with (17) ]

(8) until a stopping criterion is satisfied.

(9) output: L * , M * .

Definition 1.

For any given norm · in an inner product space, there exists a dual norm · d defined as (2) X d sup { | X , Y | : Y R m × n , Y 1 } = sup { | X , Y | Y : Y R m × n , Y 0 } . Furthermore, the dual norm of · d is again the original norm · .

From the definition, we can obtain the following upper bound of the inner product: (3) | X , Y | X d Y . Then, due to the fact that a , b > 0 always has 2 a b a + b , we can further convert (3) into the following equivalent result: (4) 2 | X , Y | X d + Y .

Thus, minimizing the sum of the norms X d and Y can get a smaller value of the inner product | X , Y | , which leads to the most irrelevant X and Y . Back to the foreground and background modeling processing in the video sequence problems, we use the nuclear norm as a prior assumption of the background. Then, if we use the dual norm of the nuclear norm to the foreground and minimize the sum of the norms in the minimization problem, then we will be able to get the most unrelated foreground and background from the known data matrix.

Theorem 2 (Proposition 2.1 in [<xref ref-type="bibr" rid="B22">22</xref>]).

The dual norm of the nuclear norm · * is the operator norm · ( it is the spectral norm · 2 ) in R m × n .

So, the minimization problem (1) can be improved as the following nuclear norm and spectral norm minimization problem (NSMP): (5) NSMP : min L , M 1 2 D - L - M F 2 + λ L * + μ M 2 , where · 2 , the spectral norm, is the largest singular value of the matrix.

This is our rudimentary model, with the fidelity term, which makes the result being faithful and loyal to the original video data; with the regularization terms, the nuclear norm ensures the low-rank of the background, and the spectral norm ensures the extraction of the foreground.

2.2. Algorithm to the Dual Norm Model

Although (5) is a nonconvex problem, while its two subproblems, nuclear norm minimization (NNM) problem and spectral norm minimization (SNM) problem, are all convex. Thus, we can solve the subproblems alternatively instead of solving (5) directly.

2.2.1. Nuclear Norm Minimization Problem

For a fixed matrix M , the NNM problem can be expressed as the following subproblem: (6) min L J 1 = arg min L 1 2 ( D - M ) - L F 2 + λ L * . The solution to this NNM problem has already been given by Cai et al. in Theorem 2.1 of his work . Suppose D - M = U Σ V T is the singular value decomposition (SVD) of the matrix D - M ; then, the solution to (6) is (7) S λ ( D - M ) : = U S λ ( Σ ) V T , where the soft-threshold operator S λ is defined as S λ : = max ( | · | - λ , 0 ) .

2.2.2. Spectral Norm Minimization Problem

Similarly, for a fixed matrix L , the other subproblem can be expressed as the following form: (8) min M J 2 = arg min M 1 2 ( D - L ) - M F 2 + μ M 2 .

We will provide the singular value threshold method for the spectral norm minimization problem based on the linear algebra in this section. For convenience, we can reexpress the problem as follows: (9) min X J = arg min X 1 2 Y - X F 2 + μ X 2 . Suppose we have the SVD of the matrix Y = U Y Σ Y V Y T ; then, we put forward the following theorem.

Theorem 3.

The solution to the minimization problem (9) can be obtained by X = U Y Γ * V Y T , where Γ * obeys (10) min J Γ * = min Γ 1 2 vec ( diag ( Σ Y - Γ ) ) 2 2 + μ vec ( diag ( Γ ) ) . vec ( diag ( · ) ) denotes the vector obtained by the rearrangement of the diagonal elements of the matrix.

Here, we assume X = U Y Γ V Y T directly and set Γ as any matrix of the same size with Σ Y and let Γ denote the diagonal of the matrix Γ . For the reason that the Frobenius norm and the spectral norm are invariant under orthogonal transformation, (9) can be firstly converted into (11) min Γ J = min Γ Σ Y - Γ F 2 + μ Γ 2 . On account of the fact that Γ 2 = Γ 2 and Σ Y - Γ F 2 Σ Y - Γ F 2 , (11) can be further converted into (12) min J Γ = min Γ Σ Y - Γ F 2 + μ Γ 2 . Finally, rewriting the above function into the vector form can get (10), which means the minimum of the solution reaches on the diagonal of the variable matrix.

The problem (10) has already been solved by Fadili and Peyre in Proposition 2 of his article . To solve the problem, we introduce a sequence, (13) d [ 1 ] d [ 2 ] d [ N ] , where { d [ t ] } t = 1 N = { σ Y [ i ] } i = 1 N , N = min ( m , n ) , means that the array { d [ t ] } is the rearrangement of the singular values of the matrix Y . And sequence (13) can be extended into (14) d [ 1 ] d [ 2 ] d [ N ] = D [ N ] D [ N - 1 ] D [ N - 2 ] D [ 2 ] D [ 1 ] , where D [ s ] = t = s N d [ t ] is the cumulated ordered values.

For μ 0 and Y R m × n , the optimal solution to the problem (9) obeys (15) T τ ( Y ) = { 0 , if σ Y 1 μ , U Y · diag ( σ Y - S τ ( σ Y ) ) · V Y T , otherwise , where (16) S τ ( σ Y ) [ i ] = max ( 1 - τ | σ Y [ i ] | , 0 ) · σ Y [ i ] , and τ > 0 is given by (17) τ = d [ t ] + ( d [ t + 1 ] - d [ t ] ) μ - D [ t + 1 ] D [ t ] - D [ t + 1 ] , where t satisfies D [ t + 1 ] μ < D [ t ] .

Such a threshold could keep most of the small singular value while changing the larger singular values as a constant, and the constant changes with the matrix Y . So, the solution to the problem (8) can be obtained by T τ ( D - L ) .

2.2.3. Alternating Iterative Algorithm

Based on aforementioned analysis, we present the alternating iterative method (AIM) to the minimization problem (5).

Stopping Criterion. We suggest using the following value to judge whether to end iteration: (18) min ( L k - L k - 1 * L k * , M k - M k - 1 2 M k 2 , D - L k - M k F D F ) ε , where ε = 1 e - 5 . And if the iterations exceed the maximum set K = 30 , the method will also stop.

Parameter λ . We set the value of λ according to different video sequence as λ = δ 1 m n , where m and n are the sizes of the data matrix D and δ 1 ( 1,10 ) is a small adjustable constant that ensures the good performance.

Parameter μ . We set μ according to μ = δ 2 n m n , where δ 2 ( 0,1 ) is also a small adjustable constant. We can see that the selection of parameters is basically multiplied by n , which is used to ensure the two regular items calculate in the same order of magnitude.

The choice of parameter values for the algorithm is very important, and there has two parameters, λ , and μ , which can be adjusted in our algorithm. Due to the slightly different parameter settings for different data, here we can only give out the setting range of these two parameters. The parameters will influence each other, thereby affecting the running of the algorithm. The selection of λ and μ cannot be too small; otherwise, the processing speed will slow down and the image will have great losses. The more complex the foreground of the frames, the greater λ and μ needed. A larger λ brings a cleaner background, and a larger μ causes a more complete extract of portrait. However, the value of the parameters can not just blindly large, too large λ will leads to the failure to extract the background, and too large μ will extract background into the foreground. The exact theoretical basis of the parameters selection will conduct a detailed study in the follow-up work.

3. Improved Dual Function Model 3.1. Model Creation

In the previous section, we put forward the preliminary dual model NSMP. And next, we will introduce an improved dual model, weighted function nuclear norm and spectral norm minimization model. This “low-rank + dual” model applies the weighted function nuclear norm to the low-rank background instead of the normal nuclear norm then find the dual representation to this weighted function nuclear norm to regular the foreground. The reason we put forward the improvement is that the well-known nuclear norm regularized problem is the convex relaxation of the rank minimization problem, but it is not a perfect approximation of the rank function. Although the minimization problem with the weighted function nuclear norm is nonconvex, fortunately it has a closed form solution due to the special choice of the value of weights, and it is also a better approximation to the rank function. Thus, what we are going to do next is introduce the weighted function and optimize our model further. By this way, we can improve (5) into the following weighted function nuclear norm and spectral norm minimization problem (WNSMP): (19) WNSMP : min L , M 1 2 D - L - M F 2 + λ w ( L ) * + μ w - 1 ( M ) 2 , where w ( · ) denotes the weighted function which directly adds the weights onto the singular values of the matrix, and, for any matrix X R m × n , weighted function norm is defined as (20) w ( X ) * i = 1 min ( m , n ) w i σ i ( X ) , (21) w - 1 ( X ) 2 max i ( 1 w i σ i ( X ) ) .

If we still rearrange the singular values σ i ( i = 1 , , min { m , n } ) of the matrix X into a vector σ , then we will have that w ( X ) * is actually the weighted l 1 -norm of the vector σ . For any vector v , there exists (22) | σ , v | = | i = 1 min ( m , n ) 1 w i σ i · w i v i | i = 1 min ( m , n ) | 1 w i σ i · w i v i | max i | 1 w i σ i | · i = 1 min ( m , n ) | w i v i | = max i | 1 w i σ i | · v w , 1 . So, we can further get (23) σ w , 1 dual = sup v w , 1 1 | σ , v | sup v w , 1 1 max i | 1 w i σ i | · v w , 1 max i | 1 w i σ i | = σ w - 1 , , where σ w - 1 , is actually the weighted function spectral norm w - 1 ( X ) 2 . Thus, we finally get that w ( X ) * and w - 1 ( X ) 2 are dual form to each other under the same weight.

3.2. Algorithm to the Dual Function Model 3.2.1. Weighted Function Nuclear Norm Minimization Problem

Here, we still use the alternating iterative algorithm to solve the problem (19). Fix the matrix M to get the first subproblem as (24) min L J 1 = arg min L 1 2 ( D - M ) - L F 2 + λ w ( L ) * .

For the convenience of discussion, we set Z = D - M and substitute (20) into the functional; then, we can rewrite (24) as follows: (25) min L J 1 = arg min L 1 2 Z - L F 2 + λ i = 1 min ( m , n ) w i σ i ( L ) . Here, we would impose increasing weight to ensure the global solution to (25), although i = 1 min ( m , n ) w i σ i ( L ) is no longer a matrix norm. In our paper, we determine the adaptive weights by (26) w i δ n ( σ i ( L ) + c ) , where δ = 1 e + 2 is an adjustable constant to ensure the value range of the weights including 1, n is the number of the frames included in the sample image sequence, σ i ( L ) denotes the singular values of the low-rank matrix L , and c = 1 e - 16 is to avoid the initial value of zero.

Theorem 4 (Theorem 2.3 in [<xref ref-type="bibr" rid="B26">26</xref>]).

For any λ 0 , Z R m × n , and 0 w 1 w 2 w min ( m , n ) , a global optimal solution to the optimization problem (25) is given by the adaptive SVD soft-thresholding (ASVT) operator L * T λ w ( Z ) , where Z = U Z Σ Z V Z T , and (27) T λ w ( Z ) U Z S λ w ( Σ Z ) V Z T , S λ w ( Σ Z ) diag { ( Σ Z [ i ] - λ w i ) + , i = 1 , , min ( m , n ) } .

Thus, the solution to the problem (24) is T λ w ( D - M ) .

3.2.2. Weighted Function Spectral Norm Minimization Problem

Fixing the matrix L , we can express WNSMP by the following subproblem: (28) min M J 2 = arg min M 1 2 ( D - L ) - M F 2 + μ w - 1 ( M ) 2 . For any μ 0 , the solution to the problem (28) is T τ / w ( D - L ) , and, for any matrix, X = U X Σ X V X T has (29) T τ / w ( X ) = { 0 , if σ X 1 μ U X · diag ( σ X - S τ / w ( σ X ) ) · V X T , otherwise , where S τ / w ( σ X ) [ i ] = ( σ X [ i ] - ( τ / w i ) ) + .

3.2.3. Alternating Iterative Algorithm

In order to enable the values of weights to be updated in each iteration process, we set w i k in the k th iteration associated with the low-rank matrix L k . But the problem is that, in the k th iterative, the matrix L k is not available. So, we need to use the matrixes Z k - 1 to estimate the value of the matrix L ^ k as (30) Σ i ( L ^ k ) = Σ i ( Z k - 1 ) = Σ i ( D - M k - 1 ) . We always rearrange the value included in the set { δ n / ( Σ i ( D - M k - 1 ) + c ) , i = 1 , , min ( m , n ) } to ensure the weights to be nondecreasing. Stopping criterion and parameter setting are the same as in NSMP.

4. Experimental Results

In this section, we present the numerical experiments corroborating our main results. We practice IALM, GoDec, NSMP, and WNSMP in a wide variety of surveillance video sequences, including five different scenarios: lobby, restaurant, airport, shopping mall, and campus, and all the datasets can be downloaded in http://perception.i2r.a-star.edu.sg/bk_model/bk_index.html. We stack each 200 frames from one scenario to be rearranged as the columns of one data matrix and use these four algorithms to decompose the data matrix into different background and foreground. All the selected frames from the five video sequences are listed in Figure 1. We can see that the foreground part in Figure 1(a) is just a single person, while, in Figures 1(b), 1(c), and 1(d), there are a lot of people in motion; further to Figure 1(e), the moving foreground is expanded into objects.

Test samples from video sequence: lobby (a), airport (b), shopping mall (c), restaurant (d), and campus (e).

Figure 2 shows us the segmentation results in the case that only one person is moving in the video, and we can see that the difference of the performance is inconspicuous, four kinds of methods all doing well.

Lobby: (a)–(d) background and (e)–(h) foreground from IALM, GoDec, NSMP, and WNSMP.

In the simple situations, the advantage of our algorithm is not very obvious, but, in a relatively complex case, the superiority will be immediately apparent. The backgrounds of IALM in Figures 3(a), 4(a), and 5(a) always exist incomplete processed moving shadows in the results. And the artifacts problem in both foreground and background of GoDec gets worse. Characterized by three kinds of situations, backgrounds always have the unnatural artifacts shown in Figure 4(b); as shown in Figure 5(b), there exist incomplete processing moving shadows in the processed background just like IALM in Figure 5(a); Even more worse, due to the interference by the other frames, unexpected shadow appears in the results of the current frame, as shown in Figure 5(f). While, in these processes, our algorithms always have stable performance with clear background and complete foreground.

Airport: (a)–(d) background and (e)–(h) foreground from IALM, GoDec, NSMP, and WNSMP.

Shopping mall: (a)–(d) background and (e)–(h) foreground from IALM, GoDec, NSMP, and WNSMP.

Restaurant: (a)–(d) background and (e)–(h) foreground from IALM, GoDec, NSMP, and WNSMP.

Figure 6 shows the far more complex situations. Although the results do not live up to the expectation of greater progress, our method still has a relatively good processing results. The backgrounds in our methods are always very clear, while GoDec has a little artifact, and IALM is even worse.

Campus: (a)–(d) background and (e)–(h) foreground from IALM, GoDec, NSMP, and WNSMP.

We can easily conclude from the observed image, Figures 26, that our model can get a better and clearer background with low-rank. Therefore, in order to show that our model is actually good enough to model the background, we display the precision-recall (PR) curves and the receiver operator characteristic (ROC) curves in Figure 7 to evaluate the binary decision problem about the decomposition of the foreground and background. Furthermore, we list the rank of the final modeling background of the four methods in Table 1.

The data comparison of background modeling and foreground detection.

Method Lobby Airport Shopping mall Restaurant Campus
128 * 160 144 * 176 256 * 320 120 * 160 128 * 160
IALM
Time 30.69 42.55 145.97 30.46 30.98
Iteration 33 35 34 34 34
Rank 111 103 91 103 101
Correlation 0.1822 0.3107 0.1900 0.2166 0.1567
GoDec
Time 40.41 61.30 182.84 56.34 66.59
Iteration max_iter max_iter max_iter max_iter max_iter
Rank 2 2 2 3 2
Correlation 0.2518 0.3867 0.2485 0.3135 0.2600
NSMP
Time 10.87 18.66 60.21 11.21 12.90
Iteration 6 8 7 7 7
Rank 3 4 5 4 5
Correlation 0.1082 0.1059 0.0935 0.1743 0.0819
WNSMP
Time 9.06 15.79 50.81 11.20 12.63
Iteration 5 7 6 7 7
Rank 3 3 4 4 4
Correlation 0.0895 0.0622 0.0517 0.0917 0.0639

PR curves and ROC curves for the background modeling of IALM, GoDec, NSMP, and WNSMP.

Under the noiseless situation, “low-rank” is the unique characteristic of the background in the video sequence, and the small correlation can separate the nonlow-rank foreground from the background. Thus, our “low-rank + dual” model not only keeps the background but also extracts the foreground part of the video. Additionally, the correlation of the foreground and background images has been given in Table 1. Here, we adopt the following ways to measure the correlation of the foreground and background images: (31) Cor ( L , M ) = max ( | l i T m i | l i 2 m i 2 ) , where l i and m i denote each column in the matrixes L and M , which are the foreground and background images from the same frame to pull into columns.

Through the observation of the visual and data results, it is easy to draw the advantages and disadvantages of each method. IALM has a relatively fast speed, but it offers a blind separation of the low-rank and sparse data. Therefore, once the foreground becomes more complex, the detection of background will be obvious interference by the artifacts. More importantly, every time the result of the low-rank approximation matrix basically has reached the rank up to around 100, which in some sense no longer can be called a low-rank matrix. GoDec has two adjustable parameters, “rank” and “card,” in the algorithm. Presetting “rank” can guarantee the approximation matrix to be necessarily low-rank, but, as the rank has been fixed already, it can not adaptively adjust according to the situation of scene. And the correct selection of “card” is also very important, which has a great influence on the results. The larger the value of “card” the greater time the algorithm runs, and the background would become messy; the smaller the value of “card” the lower time consumption, but the foreground appears lack of information. Both our methods can adaptively choose the appropriate low-rank and ensure the most irrelevant foreground and background to be separated. The parameter setting of our methods has been already stated in Section 2, and the parameters of IALM and GoDec are set as in [15, 16].

5. Discussion and Future Work

This paper proposes a “low-rank + dual” model to decompose the static background and the moving foreground in each frame of the video surveillance sequence. Our model decomposes the stack data matrix D into two parts, L low-rank background and M moving foreground, and we upgrade this model further into a reweighted function norm form. Through the experiment, we can clearly observe that our two models can not only achieve clearer backgrounds but also have the lower consumption and correlation. Our “low-rank + dual” model is a better choice for the background modeling, which can always extract clear and complete background in any case. However, since the focus of this paper is to propose the new model and show the feasibility of this model, the algorithm itself has not been optimized which still uses singular value decomposition to get the solution. So, the computation complexity of our algorithm is still O ( n 3 ) , and the space complexity is O ( m 2 ) . Using the outstanding fast SVD method or other effective methods to speed up our algorithms will be our next work. Also, the further study in the choice of the optimal parameters and weights can be a follow-up process.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors thank Zhouchen Lin and Tianyi Zhou for their respective open source codes. The authors also would like to thank anonymous reviewers for their kind comments. This paper is supported by the National Nautral Science Foundation of China (nos. 61271294, 61105011, and 61362029).

Moeslund T. B. Hilton A. Krüger V. A survey of advances in vision-based human motion capture and analysis Computer Vision and Image Understanding 2006 104 2-3 90 126 10.1016/j.cviu.2006.08.002 2-s2.0-33749990780 Toyama K. Krumm J. Brumitt B. Meyers B. Wallflower: principles and practice of background maintenance 1 Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV'99) September 1999 255 261 2-s2.0-0033285765 Stauffer C. Grimson W. E. L. Adaptive background mixture models for real-time tracking 2 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99) June 1999 Fort Collins, Colo, USA 246 252 2-s2.0-0032634283 10.1109/CVPR.1999.784637 Stauffer C. Grimson W. E. L. Learning patterns of activity using real-time tracking IEEE Transactions on Pattern Analysis and Machine Intelligence 2000 22 8 747 757 10.1109/34.868677 2-s2.0-0034244889 Lee D.-S. Effective Gaussian mixture learning for video background subtraction IEEE Transactions on Pattern Analysis and Machine Intelligence 2005 27 5 827 832 10.1109/TPAMI.2005.102 2-s2.0-18144412408 Zivkovic Z. Improved adaptive Gaussian mixture model for background subtraction Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04) August 2004 Cambridge, UK 28 31 2-s2.0-10044240378 Elgammal A. Harwood D. Davis L. S. Non-parametric model for background subtraction Computer Vision—ECCV 2000 2000 Berlin, Germany Springer 751 767 Elgammal A. Duraiswami R. Harwood D. Davis L. S. Background and foreground modeling using nonparametric kernel density estimation for visual surveillance Proceedings of the IEEE 2002 90 7 1151 1162 10.1109/JPROC.2002.801448 2-s2.0-33645307063 Oliver N. M. Rosario B. Pentland A. P. A bayesian computer vision system for modeling human interactions IEEE Transactions on Pattern Analysis and Machine Intelligence 2000 22 8 831 843 10.1109/34.868684 2-s2.0-0034245149 Chandrasekaran V. Sanghavi S. Parrilo P. A. Willsky A. S. Sparse and low-rank matrix decompositions Proceedings of the 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton '09) October 2009 962 967 10.1109/ALLERTON.2009.5394889 2-s2.0-77949622781 Candes E. J. Li X. Ma Y. Wright J. Robust principal component analysis? Journal of the ACM 2011 58 3, article 11 10.1145/1970392.1970395 MR2811000 de La Torre F. Black M. J. Robust principal component analysis for computer vision Proceedings of the 8th International Conference on Computer Vision (ICCV '01) July 2001 362 369 2-s2.0-0034849799 Wright J. Ganesh A. Rao S. Robust principal component analys is: exact recovery of corrupted low-rank matrices by convex optimization 3 Proceedings of the Neural Information Processing Systems 2009 Yuan X. Yang J. Sparse and low-rank matrix decomposition via alternating direction methods Pacific Journal of Optimization 2013 9 1 167 180 Lin Z. Liu R. Su Z. Linearized alternating direction method with adaptive penalty for low-rank representation NIPS 2011 2, article 6 Zhou T. Tao D. GoDec: randomized low-rank & sparse matrix decomposition in noisy case Proceedings of the 28th International Conference on Machine Learning (ICML '11) July 2011 33 40 2-s2.0-80053437358 Peng Y. Ganesh A. Wright J. Xu W. Ma Y. RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images IEEE Transactions on Pattern Analysis and Machine Intelligence 2012 34 11 2233 2246 10.1109/TPAMI.2011.282 2-s2.0-84866665730 Demmel J. W. Applied Numerical linear Algebra 1997 Philadelphia, Pa, USA SIAM Süli E. Mayers D. F. An Introduction to Numerical Analysis 2003 Cambridge University Press 10.1017/CBO9780511801181 MR2006500 Meyer Y. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures 2001 Boston, Mass, USA American Mathematical Society Kolmogorov A. N. Fomin S. V. Elements of the Theory of Functions and Functional Analysis 1999 Courier Dover Recht B. Fazel M. Parrilo P. A. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization SIAM Review 2010 52 3 471 501 10.1137/070697835 MR2680543 ZBL1198.90321 2-s2.0-78549288866 Cai J. Candès E. J. Shen Z. A singular value thresholding algorithm for matrix completion SIAM Journal on Optimization 2010 20 4 1956 1982 10.1137/080738970 MR2600248 2-s2.0-77951291046 Donoho D. L. De-noising by soft-thresholding IEEE Transactions on Information Theory 1995 41 3 613 627 10.1109/18.382009 MR1331258 2-s2.0-0029307534 Fadili J. M. Peyre G. Total variation projection with first order schemes IEEE Transactions on Image Processing 2011 20 3 657 669 10.1109/TIP.2010.2072512 MR2799177 2-s2.0-79951847467 Chen K. Dong H. Chan K. Reduced rank regression via adaptive nuclear norm penalization Biometrika 2013 100 4 901 920 10.1093/biomet/ast036 MR3142340 ZBL1279.62115