This paper addresses the problem of direction-of-arrival (DOA) estimation of coherent signals in the presence of unknown mutual coupling, and an autoregression (AR) model-based method is proposed. The effects of mutual coupling can be eliminated by the inherent mechanism of the proposed algorithm, so the DOAs can be accurately estimated without any calibration sources. After the mixing matrix is estimated by independent component analysis (ICA), several parameter equations are established upon the mixing matrix. Finally, all DOAs of coherent signals are estimated by solving these equations. Compared with traditional methods, the proposed method has higher angle resolution and estimation accuracy. Simulation results demonstrate the effectiveness of the algorithm.

Direction-of-arrival (DOA) estimation is very important in a variety of wireless communication applications, such as mobile communication, radar, and distributed sensor networks. In particular, many effective high-resolution DOA estimation algorithms have been developed and deeply investigated in the last decades [

In the last years, many array calibration algorithms have been proposed with respect to the mutual coupling effect [

On the other hand, many techniques have been proposed to deal with the correlated or coherent situation. A forward/backward spatial smoothing (FBSS) method that can solve the coherent problem is presented in [

However, it is more difficult to estimate DOAs of coherent signals in the presence of unknown mutual coupling. Dai and Ye [

The paper is organized as follows. The data model of the ULA is given in Section

Consider

In the presence of mutual coupling, the true steering vector should be modified as

The covariance matrix of the received signals is defined by

We assume

According to (

Because

Substituting (

The real steering vectors in the presence of unknown mutual coupling can be given by

Then we will introduce the proposed method to solve the problem of DOA estimation of coherent signals in the presence of unknown mutual coupling.

For any given column vector

For the second equation to the (

Using the same principle to process all the adjacent

In addition, as the signals are of complex value, we utilize their conjugate information to improve the precision of the proposed method as [

Combining (

From the case of

According to (

By comparing (

All roots of (

Due to (

It is not difficult to see that the number of coherent signals in one group

For the improved FBSS, the detectable number of source

When

In this section, some computer simulations are reported to illustrate the performance of our proposed method. In the following simulations, we will compare the proposed method to FBSS [

In the first simulation, we consider one group of two coherent signals impinging on an 8-element ULA from the directions

RMSE of DOA estimates against SNR (

In the second simulation, we consider the more complicated situation: two groups of two coherent signals impinge on an 8-element ULA from the directions

RMSE of DOA estimates against SNR (

In the third simulation, we will validate the high spatial resolution of the proposed method. Consider one group of two coherent signals with SNR of 10 dB impinging on an 8-element ULA, where the amplitude factors are the same as simulation 1. Assume that the directions of two coherent signals are

The RMSE of DOA estimation versus the angle interval.

To verify the maximum detectable number of source signals by the proposed method, in the forth simulation, we consider five groups of two coherent signals impinging on a 5-element ULA from the directions

The RMSE of DOA estimation against SNR (

In this paper, an AR model-based DOA estimation algorithm is proposed for coherent signals in the presence of unknown mutual coupling. The effects of mutual coupling can be eliminated by solving a mathematical equation. Simulation results demonstrate that the proposed method has high spatial resolution and DOA estimation accuracy compared to the improved FBSS algorithm. Furthermore, the number of signals resolved by our method is larger than that of others.

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions which vastly improved the content and presentation of this paper. This research was supported by the National Natural Science Foundation of China (no. 61072120) and the Program for New Century Excellent Talents in University of China (NCET).