Coherent Target Direction-of-Arrival Estimation for Coprime Arrays: From Spatial Smoothing Perspective

In this paper, we investigate the issue of direction-of-arrival (DOA) estimation of multiple signals in coprime arrays. An algorithm based onmultiple signal classification (MUSIC) and forward and backward spatial smoothing (FBSS) is used for DOA estimation of this signal caused by multipath and interference. The large distance between adjacent elements of each subarray in the coprime arrays will bring phase ambiguity issues. According to the feature of the coprime number, the ambiguity problem can be eliminated. The correct DOA estimation can be obtained by searching for the common peak of the spatial spectrum and finding the overlapping peaks in the MUSIC spectrum of the two subarrays. For the rank deficit problem caused by the coherent signal, the FBSS algorithm is used for signal preprocessing before the MUSIC algorithm. Theoretical analysis and simulation results show that the algorithm can effectively solve the rank deficiency and phase ambiguity problems caused by coherent signals and sparse arrays in the coprime arrays.


Introduction
Array signal processing is a branch of the signal processing field and is widely used in radar, sonar, satellite, wireless communications, seismology, and other fields [1,2]. Array signal processing is based on a group of spatially arranged array antennas to process the signal [3]. The purpose of array signal processing is to enhance useful target signals, suppress noise, and obtain signal spatial information. Compared with a single antenna, the use of an antenna array has outstanding advantages in terms of spatial resolution, receiving sensitivity, and anti-interference [4]. Thus, array signal processing has made rapid progress in research and engineering applications in the past 30 years [5].
DOA estimation of space signal is a basic problem in array signal processing. DOA estimation is to estimate the direction of arrival of the signal by receiving the target echo data through the array antenna in the noise or interference environment. And it is a kind of direction-finding technique [6,7]. In wireless communication, accurate DOA estimation of the signal source can improve communication quality [8]. And it can improve physical layer security combined with beamforming technology [9]. In radar target detection, DOA estimation is the basis for achieving high-precision direction finding [10]. Therefore, it is of great significance to study how to improve the accuracy of DOA estimation. The performance of DOA estimation is determined by the resolution, accuracy, the number of distinguishable targets, etc. [11]. In response to these aspects, corresponding theoretical and applied research has been carried out at home and abroad, which has enabled the rapid development of DOA estimation theory [12][13][14].
The past researches have proposed a large number of DOA estimation algorithms for different array models, such as the uniform linear array, L-shaped linear array, and uniform circular array [15][16][17]. In the traditional array structure, the angle ambiguity is avoided by setting the spacing of array elements no more than half wavelength. However, when the frequency of the received signal is high, too small array element spacing will cause larger mutual coupling, and the physical array layout is difficult to achieve. At the same time, high resolution means a larger array aperture, and more physical array elements will further increase the system cost and complexity. Sparse arrays can overcome the structural limitations of traditional arrays by increasing the array element spacing [18] and thus have been developed and widely used, such as the Minimum Redundancy Array (MRA) [19,20], Nested Array (NA), and coprime array (CPA) [21][22][23].
The coprime formation is composed of two subarrays, and the spacing between the subarrays is mutually prime [24]. Compared with the traditional uniform array, the element spacing of the coprime array is greater than half a wavelength. The increase in the element spacing brings the advantages of an increase in the array aperture and a significant reduction in the mutual coupling effect between elements and significantly improves the estimation accuracy and resolution [25]. The DOA of two uniform subarrays of the coprime array is estimated, respectively. According to the relatively prime characteristics of the element spacing of the two subarrays, it is proved that the DOA estimation results of the two subarrays are unique [26,27]. The coprime array which does not reduce the array aperture of the original array is simple to implement, and the estimation accuracy is greatly improved compared with the uniform array with the same number of antennas [28][29][30].
DOA estimation algorithms mainly include traditional beamforming, subspace algorithm, and maximum likelihood estimation [31][32][33]. Among them, the beamforming method has larger error and low resolution; the maximum likelihood algorithm uses the probability distribution of the signal and adopts the high-dimensional search method, which has a large amount of computation. The subspace algorithm uses the orthogonality of signal and noise subspace to realize angle estimation, which requires less computation but cannot process coherent signals [34]. Generally, the minimum resolution that can be achieved under a certain array length is called the Rayleigh Resolution Limit, and the method that exceeds the Rayleigh Resolution Limit is called the superresolution algorithm. Multiple signal classification (MUSIC) proposed in 1979 and estimating signal parameters via rotational invariance techniques (ESPRIT) proposed in 1986 belong to subspace algorithm and are also early classical superresolution methods [35,36]. No matter the MUSIC algorithm or ESPRIT algorithm, it is necessary for the array element to receive the uncorrelated signal. At this time, the covariance matrix of the source is a full rank matrix, so that the covariance matrix of the signal can be eigendecomposed and the signal subspace and noise subspace can be distinguished.
Most signals are coherent signals in the actual application environment because of the multipath effect and complex transmission channel [37,38]. For early DOA estimation algorithms such as MUSIC and ESPRIT, they are all based on subspace for DOA estimation. When the received signal is correlated, the eigenvector corresponding to the source signal cannot be obtained by decomposing the subspace eigenvalues. Therefore, DOA estimation of coherent source signals has always been a difficult problem, which is also the focus of spectral estimation. In order to distinguish coherent signals accurately, the spatial smoothing method, singular value decomposition method (SVD method), matrix decomposition method (MD method), and Toeplitz method are developed [39][40][41].
In this paper, the MUSIC algorithm and the FBSS algorithm are combined to estimate the DOA of coherent signals based on the coprime matrix model under the condition of multipath and interference, and the formulas to solve the signal coherence and angle ambiguity under the coprime matrix are given. Finally, the DOA estimation method for coherent signals is simulated, and the simulation results show the effectiveness of the method.
The remainder is given as follows: Section 2 outlines the basic array signal model of the coprime array. In Section 3, the proposed method for coherent target DOA estimation based on coprime arrays is presented, and the problem of phase ambiguity and rank deficiency is discussed together with its elimination method. Numerical simulations and conclusions are presented in Sections 4 and 5, respectively.
Notations. Throughout the paper, we use the lowercase (uppercase) boldface symbols to represent vectors (matrices). ð⋅Þ T and ð⋅Þ H denote the transpose and the conjugate transpose, respectively. I N denotes N × N identity matrix, diag ð⋅Þ denotes the diagonal matrix operator, and E½⋅ denotes the expectation operator.

Array Signal Model
The coprime array is a sparse array constructed by using the property of a coprime number. It is composed of two uniform linear arrays. Assuming that the number of subarray elements is M and N, the spacing between two subarrays is Nλ/2 and Mλ/2, respectively, where M and N are coprime integers and λ represents the wavelength of the received signal. The first element of the two subarrays coincides, which is also called the reference element. The coprime array contains a total of elements, and the positions of M + N − 1 elements, and the positions of the elements are where 0 ≤ m ≤ M − 1, 0 ≤ n ≤ N − 1: Figure 1(a) is a schematic diagram of the structure of a coprime array with M + N − 1 elements. For the convenience of analysis, the coprime matrix is divided into two subarrays, in which the black dot represents subarray 1 and the hollow dot represents subarray 2, as shown in Figure 1(b). In fact, the two subarrays are in a straight line and share the first element.
It is assumed that there are far-field narrow-band signals from different directions in the space, the incident angle is θ k , k = 1, 2, 3, ⋯, K, and the output noise of each element is a complex Gaussian distribution with zero mean value, which are independent of each other and have the same average power σ 2 . The output of the mth element can be expressed as

Wireless Communications and Mobile Computing
If the first element is selected as the reference element, the output of the subarray with M elements is where The steering vector of the Kth source is expressed as Source vector sðtÞ = ½s 1 ðtÞ, s 2 ðtÞ, ⋯, s K ðtÞ T . Similarly, the output of the subarray with N elements is where A N ðθÞ = ½a N ðθ 1 Þ, a N ðθ 2 Þ, ⋯, a N ðθ K Þ. The steering vector of the Kth source is expressed as Because the noise is independent of each other, the noise and the signal are independent of each other, the covariance matrix of the noise is σ 2 I, and the covariance matrix of the output of the two subarrays: In the DOA estimation based on the coprime array, the array aperture is greatly expanded by the construction of a virtual array model. At the same time, the ranks of covariance matrices constructed by different methods are also different, but generally, the virtual array degree of freedom of the coprime array is far greater than that of the physical array. The degree of freedom is an important sign that the antenna array can estimate the number of targets or sources. The higher the degree of freedom is, the more sources the array can estimate. Besides, the degree of freedom is proportional to the estimation accuracy. Generally, the higher the degree of freedom is, the higher the positioning accuracy will be.

DOA Estimation of Coherent Signals
Due to the interference effect of coherent signals, the number of subspaces processed by the ordinary DOA estimation algorithm will be reduced and affect the direction-finding accuracy. However, the FBSS algorithm does not appear in such a situation. Based on this idea, we decompose the coprime array into two uniform subarrays. For each subarray, the FBSS algorithm and the traditional MUSIC algorithm are combined to process the coherent signal. By analyzing the DOA results of the two subarrays, the correct target angle can be obtained, and the problem of rank deficiency caused by phase ambiguity and coherent signal is solved.

Spatial
Smoothing on Subarrays. Coherent signals are easily generated in signal transmission due to the complex space environment. The appearance of coherent sources may lead to serious degradation of DOA estimation performance. In the traditional MUSIC algorithm based on subspace, the covariance matrix of the received data needs to be full rank, but the covariance matrix of the coherent source is not full rank, the signal eigenvectors diverge into the noise subspace, and the singular value decomposition cannot completely distinguish the signal subspace from the noise subspace, which leads to deterioration of DOA estimation performance.
The basic idea of the spatial smoothing algorithm is to divide the array into several overlapping subarrays and use the covariance matrix of the received data of subarrays to replace the original covariance matrix. By sacrificing a certain effective array aperture, the covariance matrix of the received data is restored to full rank, so as to achieve the preprocessing operation of decoherence.
The covariance matrix uses the autocorrelation relationship between signals to extract information. In practical applications, the maximum likelihood function of the covariance matrix is usually calculated by selecting a large enough number of snapshots to approximate the ideal covariance matrix. In this case, the estimated covariance matrix of the output data can be expressed as

Wireless Communications and Mobile Computing
As shown in Figure 2, consider the subarray with the number of M elements and the spacing of Nλ/2 in the coprime matrix. Under the forward space smoothing algorithm, the equidistant linear array is divided into L subarrays by sliding, and each subarray has n elements, where n = M − L + 1.
In this case, the output of the first forward subarray can be expressed as where A M = ½a M ðθ 1 Þ, a M ðθ 2 Þ, ⋯, a M ðθ K Þ is the n * K dimension direction matrix and a M ðθÞ is the n-dimension guidance vector. D = diag ðe jð2πNd/λÞ sin θ 1 , e jð2πNd/λÞ sin θ 2 , ⋯, e jð2πNd/λÞ sin θ K Þ is a rotation-invariant matrix between subarrays.
The covariance matrix of the lth forward submatrix can be expressed as The forward spatial smoothing covariance matrix is defined as Similarly, if the subarray is divided from the last element of the array, the covariance matrix of backward spatial smoothing can be obtained as follows: Because the backward smooth array is the conjugate reverse order of the forward smooth array, the relation between R b and R l is the conjugate reverse order invariant. Although the one-way smoothing algorithm can solve the problem of coherent signals, it sacrifices more array aperture. The FBSS algorithm can increase the number of estimable cells by simultaneously performing forward and backward smoothing. The covariance matrix is the average of forward smoothing and backward smoothing covariance matrices 3.2. DOA Estimation of Subarrays. The MUSIC algorithm is the most classic superresolution DOA estimation algorithm, which obtains the cell direction by searching the spectrum peak in the spatial domain. Compared with multidimensional algorithms such as maximum likelihood (ML) and weighted subspace fitting (WSF), the algorithm has less computation. The basic idea of the MUSIC algorithm is to eigendecompose the covariance matrix of the array output data to obtain the signal subspace corresponding to the signal component and the noise subspace orthogonal to the signal component and then use the orthogonality of the two subspaces to estimate the signal parameters. The covariance matrices of the two submatrices are eigendecomposed, respectively, to obtain Among them, matrices U SM ∈ ℂ M * K and U SN ∈ ℂ N * K are the signal subspaces formed by the eigenvectors corresponding to K large eigenvalues inR M andR N , matrices ∑ SM ∈ ℂ K * K and ∑ SN ∈ ℂ K * K are the diagonal matrices formed by K large eigenvalues inR M andR N , matrices U NM ∈ ℂ M * ðM−KÞ and U NN ∈ ℂ M * ðN−KÞ are the noise subspaces formed by the eigenvectors corresponding to M − K and N − K small eigenvalues inR M andR N , and the matrices ∑ NM and ∑ NN are diagonal matrices composed of M − K and N − K small eigenvalues in R M andR N , respectively (these small eigenvalues are equal, which is the noise power σ 2 ).
Under ideal conditions, the signal subspace U S and the noise subspace U N are orthogonal to each other, so the array flow pattern vector a H ðθÞ corresponding to the signal subspace is also orthogonal to the noise subspace U N , namely, In practice, the steering vector and the noise subspace cannot be completely orthogonal due to the existence of other noises. Usually, the minimum optimization search process is used to find the minimum value to realize the direction-ofarrival estimation. This process can be expressed as Based on the orthogonality between the signal subspace and the noise subspace, the spectral function of the MUSIC space power spectrum of the two subarrays can be expressed as Among them, the value θ range is generally ð−π/2, π/2Þ.

Wireless Communications and Mobile Computing
The accurate DOA estimate can be obtained by searching for the coincident peaks of the two subarray spectral functions of the coprime array.

Ambiguity Elimination.
The calculation formula of phase defuzzification for incoherent sources is given in Reference [31]. For coherent sources, if there is phase ambiguity after using the spatial smoothing algorithm for the subarray with M elements, it can be seen from (13) that the steering vector between the real angle θ k and the blurred angle θ k should be equal, that is, After simplification, we get where P M is a nonzero integer, θ k , θ k ∈ ð−π/2, π/2Þ. For any θ k and θ k , it must satisfy jsin ðθ k Þ − sin ðθ k Þj < 2, that is, j2P/Nj Consider that θ k and θ k can be exchanged. In addition to the real angle, there are N − 1 fuzzy angles. That is to say, for a single subarray M whose element spacing is Nλ/2 in the coprime array, there must be phase ambiguity. There are N peaks in the MUSIC spectrum using spatial smoothing, and the N − 1 peaks correspond to the fuzzy angle.
In the same way, when considering the single subarray N whose element spacing is Mλ/2 in the coprime array, the fuzzy angle needs to meet the requirement: The value range of P N can be −ðM − 1Þ, −ðM − 2Þ, ⋯,−1, Combined with (19), the condition of phase ambiguity is obtained as follows: After simplification, we can get NP N = MP M . Since M and N are relatively prime, it cannot make the equation hold in the range of value; that is to say, θ k does not exist and there is no angle ambiguity. Therefore, the unique DOA estimation can be determined by using the spatial smoothing algorithm and MUSIC algorithm, respectively, for the subarrays of the coprime array, and then finding the overlapped peaks in the two groups of spectrum.
3.4. Complexity Analysis. The spatial smoothing algorithm, SVD algorithm, and Toeplitz algorithm can process coherent signals well, and the computational complexity of these three decoherence algorithms increases gradually. At present, the spatial smoothing algorithm has the least amount of computation; that is, the time of DOA processing is the shortest. In addition, spatial smoothing technology is also more mature, which is a more practical algorithm for processing coherent signals. The uniform linear array with M elements can distinguish 2M/3 coherent targets by using the spatial smoothing algorithm. And the virtual element number of the coprime array with two subarray elements M and N is OðMNÞ. Therefore, Oð2MN/3Þ coherent targets can be distinguished by the coprime array with this algorithm. In the same case, only O ð2ðM + N − 1Þ/3Þ coherent targets can be distinguished by the uniform linear array with this algorithm.

Simulation Results
In this section, we have carried out the corresponding simulation analysis to prove the effectiveness of FBSS and MUSIC algorithms for coherent signals under the coprime array model. In the simulation process, the number of elements of two subarrays of the coprime array is M = 7 and N = 5, and the spacing between elements is 5λ/2 and 7λ/2, where λ is half wavelength. For a fair comparison, a 12 uniform linear array with half-wavelength spacing is also simulated with the FBSS and MUSIC algorithms. Consider two coherent signals in the space, which are incident from 0°and 30°to the 4.1. Spatial Spectrum. We then show the spatial spectrum using FBSS and MUSIC algorithms in Figure 3, where we assume the signal to noise (SNR) as 10 dB and snapshot n = 200. The red spectral line is the subarray spectrum with M = 7, and the blue spectral line is the subarray spectrum with N = 5. It can be seen from the previous derivation that phase ambiguity will be generated when using spatial smoothing and MUSIC algorithm for a single subarray of coprime array. For a subarray with an element spacing of Nλ/2, estimating a DOA will produce N − 1 ambiguity angles. Therefore, the subarray with M = 7 has 10 peaks, 8 of which are fuzzy angles. And the subarray with N = 5 has 14 peaks, 12 of which are fuzzy angles. However, the common spectral peak formed by the two subarrays is only at 0°a nd 30°, which proves the correctness of the algorithm for DOA estimation of coherent signals.
Under this condition, we further compare the DOA estimation spectrum of 11 elements uniform linear array and coprime array. The specific results are shown in Figure 4. Through the comparison of DOA estimation spectrum peaks, we can intuitively find that two coherent signals, whether uniform linear array or coprime array, can be well distinguished. But the coprime array is better than the uniform linear array in suppressing interference. Because the number of virtual elements of the coprime array is much larger than that of the uniform linear array, it has a higher degree of freedom and better estimation performance.
where N 0 denotes the times of Monte-Carlo simulations and θ ðiÞ k and θ k are the estimate and real values of the k th DOA for the nth trial, n = 1, 2, 3, ⋯, N. The targets are located at θ 1 = 10°, θ 2 = 20°, θ 3 = 30°, and θ 1 , θ 2 are coherent signals. For each simulation scenario, S = 500 rounds of Monte-Carlo runs are conducted. The Cramer-Rao bound (CRB) is plotted as a benchmark. Figure 5 depicts the RMSEs of different configurations in terms of SNR, where the number of snapshots is 200. In Figure 6, we compare the RMSEs of the two arrays versus the number of snapshots, where the SNR is set as 5 dB. It is    Wireless Communications and Mobile Computing obvious that the performance of all these configurations improves with the increase of the SNR and number of snapshots. But the performance of the coprime array is better than that of the uniform linear array in any case. Even in the case of low snapshot number and low signalto-noise ratio, the coprime array can also show good DOA estimation performance.
Although the phase ambiguity of a single subarray of the coprime array is caused by the large element spacing, accurate DOA estimation can be achieved by comparing the peak values of the two subarrays. Through the simulation experiment, we can see that compared with the uniform linear array, the coprime array using the spatial smoothing algorithm greatly improves the resolution and reduces the computational complexity.

Conclusions
In this paper, we use FBSS and MUSIC algorithms for DOA estimation of coherent signals based on the structure of coprime arrays, where the spatial spectrum of each decomposed subarray can generate spectral peaks at the actual DOAs and multiple ambiguous DOAs simultaneously. And we solve the phase ambiguity by finding the common spectral peaks in the spectrum of the two subarrays. Theoretical analysis and simulation results show that the algorithm can effectively process DOA estimation of coherent signals, and the coprime array has better performance than the uniform linear array. However, some spatial degrees of freedom are sacrificed when using the spatial smoothing algorithm. How to increase the spatial degrees of freedom and improve the direction-finding accuracy under low snapshot numbers will be our further research direction.

Data Availability
Data will be made available on request.

Conflicts of Interest
The authors declare that they have no conflicts of interest.