Adaptive Complex-Valued Independent Component Analysis Based on Second-Order Statistics

This paper proposes a two-stage fast convergence adaptive complex-valued independent component analysis based on secondorder statistics of complex-valued source signals.The first stage constructs a cost function by extending the real-valued whiten cost function to a complex-valued domain and optimizes the cost function using a complex-valued gradient. The second stage uses the restriction that the pseudocovariance matrix of the separated signal is a diagonal matrix to construct the cost function and the geodesic method is used to optimize the cost function. Compared with other adaptive complex-valued independent component analysis, the proposed method shows a faster convergence rate and smaller error. Computer simulations were performed on synthesized signals and communications signals. The simulation results demonstrate the validity of the proposed algorithm.


Introduction
Blind source separation (BSS) is the separating of a set of source signals from a set of mixed signals without the aid of information (or with very little information) about either the source signals or the mixing process.Independent component analysis (ICA) is an attractive approach for solving blind source separation problems.ICA can be divided into realvalued ICA and complex-valued ICA according to the mixed signals.Complex-valued ICA is widely used to estimate the mixing matrix or to separate complex-valued mixed signals, such as frequency domain signals [1,2], digital communication signals [3,4], functional magnetic resonance imaging signals [5], and power system signals [6].
In contrast, ICA methods in the first and second categories are not suitable for use with complex Gaussian noncircular signals.
The major advantage of SUT is that "whenever applicable, remains perhaps the simplest and most accessible approach" [24].SUT is a batch algorithm and cannot be used to process signals in real time, so some adaptive complex-valued ICA algorithms have been proposed based on second-order statistics [2,[20][21][22].Compared with other complex-valued ICA strategies, adaptive complex-valued ICA algorithms based on second-order statistics are simpler in structure and do not require the probability density of the real and imaginary parts of a complex-valued source signal to be non-Gaussian.The Scott method [20] proposes an updating formula of the separating matrix for adaptive complex-valued ICA without mathematical speculation.The Cong method [2] simultaneously uses diagonal covariance and pseudocovariance noncircular signals as the cost function to deduce the adaptive complex ICA.The convergence condition of the Scott and Cong methods requires that the covariance and pseudocovariance of the separated signal are simultaneously diagonal.For example, if only the covariance of the separated signal is diagonal, the method is unable to reach convergence until the pseudocovariance is also diagonal.This requirement could affect convergence speed.The Yang method [22] uses a two-step serial updating method to make the separated signals satisfy the above convergence condition.In the second step, Yang uses the orthogonal method to force the separating matrix to be a unitary matrix.This changes the updating direction of the separating matrix and leads to slow convergence speed.
To increase the rate of convergence, a fast complex-valued ICA method is proposed in this work.The proposed method first extends the real-valued whitening process to a complexvalued domain to provide unit variance for the processed signal.Second, this work uses the restriction that the pseudocovariance matrix of the separated signals is a diagonal matrix to construct cost function and optimize the cost function using the geodesic method.This avoids computing the square root and inverse of the separating matrix and also keeps the separating matrix to be an orthogonal matrix, without any forcing operation.This improves the convergence speed of the proposed method compared to the other adaptive methods.

Complex-Valued ICA and
Second-Order Statistics 2.1.Complex-Valued Linear ICA Model.Generally, a linear complex-valued ICA model that is noise-free can be expressed as follows: where  = [ does not affect the shape of the estimated source signal waveform, which contains most information about source signals.

Second-Order Statistics of Complex-Valued Signals.
Assume a complex-valued random column vector  =   +  , where   and   are the real and imaginary part of , respectively, and  = √ −1.The expectation [⋅] of the random vector  is defined as follows: Its covariance matrix cov() is defined as follows: where (⋅)  denotes the Hermitian transpose.Its corresponding pseudocovariance matrix is defined as follows: where (⋅)  denotes the matrix transpose.The covariance matrix together with the pseudocovariance matrix is the full expression of second-order statistics [19].If the pseudocovariance matrix equals zero, the random vector is considered circular or proper.If both the covariance matrix and pseudocovariance matrix of the random vector are diagonal with nonzero diagonal elements, the random vector is noncircular or improper, and components of the random vector are called strong uncorrelated components.

2.3.
Complex-Valued ICA Based on SUT.For any complex random vector , if the vector can be transformed into a random vector  by use of a nonsingular square matrix , where  = [ 1 ,  2 , . . .,   ]  =  has covariance that is a unit matrix and pseudocovariance that is a diagonal matrix with diagonal elements between zero and one, then the matrix  is called SUT.If the observed signal is the complex random vector  and the source signal is , then the SUT is the separating matrix in complex-valued ICA.The procedure for complex-valued ICA based on SUT is as follows [18].
(1) Whitening the complex-valued observed signals : the whitening procedure is given by where the whitening matrix  is the inverse of the matrix square root of the covariance matrix and  is the whitened signal with a unit covariance matrix.
(2) Determining the separating matrix of the whitened signal by use of Takagi's factorization: this is done according to From ( 5) and ( 6) we obtain the separating matrix  =   .

Proposed Adaptive Complex-Valued ICA
In this section, we describe an adaptive fast convergence complex-valued ICA algorithm based on second-order statistics, used in the SUT method.This is unlike other adaptive complex-valued ICA methods that simultaneously force separated signals to comply with second-order statistics.Instead, this method uses an adaptive serial updating method to realize the SUT.First, we use an adaptive method to whiten the observed signals.The cost function used in real-value whitening is directly extended to the complex-valued signal.
The cost function is given as follows: where  is the whitening matrix and   is the th whitening signal.In complex-valued signal processing, the steepest descent direction of cost function (7) is where  is the observed signal,  = ,   / * = , and   / * = 0. To avoid computing the matrix inverse, a complex-valued natural gradient is used to simplify (8): So, adaptive whitening can be expressed as follows: If we use the instantaneous value instead of the expected value in (10), we obtain the adaptive real-time whitening method: Second, we must modify the separated signals to satisfy a diagonal pseudocovariance matrix while keeping the covariance matrix as a unit matrix.We use the cost function in [22], which can be expressed as follows: where  = V, V is the separating matrix of the whitened signals and Λ is the diagonal matrix of [  ].The ordinary gradient with V * is as follows: The update of V can be written as follows: where  = [  ] is the correlated matrix of the whitened signal.At the convergence point, the pseudocovariance matrix of the separated signal is diagonal.To keep the covariance matrix of the separated signal as a unit matrix, the separating matrix V must be a unitary matrix.In [22], they directly used the method of fixed-point fastICA to force the separating matrix to be a unitary matrix: This approach has two major drawbacks.One is that (16) changes the steepest gradient direction in every iteration, which slows the convergence speed.The second is that (16) must compute the square root and the inverse of the separating matrix in every iteration, which increases the algorithm computation complexity, slowing the time of convergence.
To overcome this problem, we use a geodesic method to search the optimized separating matrix V.The geodesic method causes the separating matrix to move on the surface of the orthogonal matrix to converge to a local minimum without a forcing operation.The geodesic method is given by where If V() is a unitary matrix, then V(+1) is also a unitary matrix.By using the geodesic method, we do not need additional operations to make the separating matrix be an orthogonal matrix and change its search direction.
Using the geodesic method with self-tuning [26] to optimize the cost function (12), we can describe a fast convergence complex-valued ICA method.The implementation process of the proposed adaptive ICA method is as follows: (1) Initialize the whitening matrix and separating matrix using unit matrix, learning rate  1 and  2 , and iterative number for optimizing ( 7) and ( 12), respectively.
(2) Use (10) to whiten the observed signal  and obtain the whitening signal  =  and whitening matrix .
(3) Compute the gradient of the cost function in Riemannian space, which can be expressed as follows: where Λ is a diagonal matrix with diagonal elements ,  = V()V()  ,  = [  ], and  = V() *   .

Experimental Results and Analysis
In order to test the algorithm, we used five synthesized signals with different spectral coefficients, three digital communication signals with different spectral coefficients, and three synthesized signals of which two signals have same spectral coefficients as the source signals.For simplicity, we directly used the expectation of the signal instead of the instantaneous value.Quality of separation was assessed using the performance index (PI), a widely used index in ICA.PI can be expressed as [27] PI where ℎ  is the (, ) element of the global system matrix  = ,  is the separating matrix of mixed signal,  is the mixing matrix, and max  |ℎ  | and max  |ℎ  | are the maximum absolute value of the elements in the  row and  column vector , respectively.When perfect separation is achieved, the performance index is zero."In practice, the value of performance index 10 −2 gives quite a good performance" [27].
The smaller the value of PI, the better the performance.In the first experiment, five complex-valued synthesized source signals with 10000 samples were used, constructed as follows: where  = 1, 2, 3, 4, 5,  (0,) () is a sample drawn from a normal random distribution within (0, ), and  = √ −1.The mixing matrix is a complex-valued random matrix with real and imaginary parts generated from a random uniform distribution between 0 and 1.All algorithms have the same learning rate of 0.01 and were run 100 times.Each time, the source signal and mixing matrix was independently generated.
In contrast, convergence curves are shown in Figure 1 that correspond to the four methods: Yang method [22], Scott method [20], SUT method [18], and our proposed method.Every method has 100 convergence curves, and every convergence curve corresponds to results from one run.The SUT method is a batch method without iterative computations.Therefore, the convergence curves are straight lines.From Figure 1, we can see that all the convergence curves of the proposed method are more closer than the other adaptive methods except for the SUT method.This suggests that the proposed method shows improved, stable performance for different mixed sources that is better than the other adaptive methods.The SUT method shows the smallest fluctuation range, followed by the proposed method, Scott method, and then the Yang method.This indicates that the proposed method is more suitable for processing different mixed signals than the other adaptive methods, except for the SUT method.Although the performance of SUT is more stable than the other methods for separating different mixed signals, its realization involves Takagi's factorization that is difficult to implement and is not suitable for real-time separation of mixed signals.The adaptive complex-valued BSS method is easy to perform and is more appropriate for real-time separation of mixed signals.
Average convergence curves for the four methods are shown in Figure 2. From Figure 2, we see that the Yang method does not converge to a stationary point until 40000 iterations; the Scott method starts to converge close to 35000 iterations; the proposed method starts to converge after about 2000 iterations.Thus, the proposed method has a faster convergence speed.The performance index is larger than the proposed method when the Scott method converges to a stationary point.This indicates that the proposed method has a smaller error than the Scott method.The performance  indices of the proposed method and the SUT method are very similar, indicating that the two methods have almost the same amount of error.
In the second experiment, we supposed that three digital communication signals (8QAM, 4QAM, and BPSK) impinge on a uniform linear antenna array with three elements from directions of 10 ∘ , 25 ∘ , and 70 ∘ .In Figure 3, the first row gives the original source signals, the second row gives the three mixed signals that are separately received by the three elements of antenna, and the third row provides the separated signals obtained using the proposed method.Comparing the source signals with the separated signals, we see that the constellation of separated signals is almost the same as the source signals, except the sequence, amplitude, and phase, which are inherently indeterminate.This shows that the proposed method is valid for the supposed communication signals.
The average convergence curves for the four methods are shown in Figure 4 from an average of 100 different simulation runs with a learning rate of 0.01.From Figure 4, we see that the proposed method starts to converge after 150 iterations, the Scott method starts to converge after 4500 iterations, and the slowest to converge is the Yang method, which starts to converge after 17000 iterations.Thus, the proposed method has faster convergence than the other adaptive methods.When the proposed method convergences to the stationary point, the performance index curve of proposed method and SUT method are the same.This means that the two methods have the same error for the communicating signals.
In the third experiment, three random complex-valued signals were used as source signals, with spectral coefficients of 0, 0.6, and 0.6.Their imaginary and real parts were generated by a random uniform distribution function.Average convergence curves from an average of 100 different simulation runs with a learning rate of 0.01 are shown for the four methods in Figure 5. From Figure 5, we see that the performance indexes of Yang method, the Scott method, and the proposed method are less than 0.1 at the stationary point.
The average performance index of SUT is about 0.33, which is far greater than 0.1.According to [27], this means that the three adaptive methods successfully separated the mixed signals but the SUT method failed for the mixed signals.
The SUT method includes Takagi's factorization to factorize the pseudocovariance matrix.Therefore, it is not suitable for noncircular signals with the same spectral coefficients.
The proposed method has two stages.The convergence curves shown in all figures are the convergence curves only for the second stage.For the first stage, the whitening signal converges to the unit matrix in first experiment after about 600 iterations and after about 100 iterations in the second and third experiments.Compared with other methods, the total iterations required for the proposed method are far less than other methods.

Conclusions
This paper proposes an adaptive complex-valued ICA method for noncircular signals based on second-order statistics and the geodesic method.The proposed method has faster convergence and smaller error than the other adaptive methods.For different mixing source signals, the proposed method has better performance and faster convergence than the Scott method.For source signals with different spectral coefficients, the proposed method and the SUT method have almost the same error.However, the SUT method is not suitable for source signals that some of source signals have the same spectral coefficients.

Figure 1 :
Figure 1: Convergence curves of four methods with synthesized signals.

Figure 2 :
Figure 2: Average convergence curves of four methods with synthesized signals.

Figure 3 :
Figure 3: Original signals, mixed signals, and separated signals in the digital communication system.

Figure 4 :
Figure 4: Average convergence curves of the four methods for digital communication signals.

Figure 5 :
Figure 5: Average convergence curves of all methods for source signals; two of these signals have the same spectral coefficients.
1 ,  2 , . . .,   ]  is the unknown column vector of source signals,  is the number of source signals,