Limiting Spectral Distribution of Large-Dimensional Sample Covariance Matrices Generated by the Periodic Autoregressive Model

)e explicit representation for the limiting spectral moments of sample covariance matrices generated by the periodic autoregressive model (PAR) is established. We propose to use the moment-constrained maximum entropy method to estimate the spectral density function.)e experiments show that the maximum entropy spectral density function curve obtained based on the fourth-order limiting spectral moment can match histograms of the eigenvalues of the covariance matrices very well.


Introduction
Periodic time-series models are used for modeling stationary time series with periodic characteristics. Periodicity refers to wave or oscillating movements around long-term trends presented in the time series, such as the water sector, where the supply of water varies with seasonal rainfall. In the economic time series, it is often accompanied by some cycle or quasi-periodic fluctuations, such as day and night changes of urban electricity consumption, seasonal changes of railway passenger transport, and repeated acceleration, interruption, stagnation, and recovery of economic growth; see Franses and Paap [1], Bell et al. [2], and Aliat and Hamdi [3]. e periodic autoregressive (PAR) model is a generalisation of the classical autoregressive (AR) model by allowing the parameters to vary with the variables over a period: For a given v and a predetermined value p, the random variable X ip+v denotes the value in the vth quarter of the ith cycle, where v ∈ 1, . . . , p and i ∈ 1, . . . , n { }. In the ith cycle, X ip+v is an AR process. e model order is given by q(v), and α k (v), k � 1, . . . , q(v), represents the AR model coefficients for the vth quarter. e error process ε � ε ip+v corresponds to a periodic white noise, with E(ε ip+v ) � 0 and var(ε ip+v ) � σ 2 (v) > 0. Let X i � (X ip+1 , . . . , X ip+p ) ⊤ , where the transpose (indicated by ⊤) of a row vector is a column vector, and we define the large-dimensional sample covariance matrices as B n � (1/n) n i�1 X i X ⊤ i , with the periodic length p tends to infinity in proportion to the sample size n, namely, p/n ⟶ c ∈ (0, ∞). Sample covariance matrices are very important in multivariate statistical inference since many test statistics are defined by their eigenvalues or functions.
Random matrix theory (RMT) originated from the study of the energy levels of a large number of particles in quantum mechanics. Many of the laws of mathematical physics were discovered through numerical studies. In the late 1950s, E. P. Wigner formulated the problem in terms of the empirical distribution of random matrices [4,5], which led to the study of the semicircular law of Gaussian matrices. Since then, RMT has formed an active branch in modern probability theory. e LSD of the sample covariance matrix of a largedimensional random matrix composed of independent random variables, the MP law, was first established by Marcenko and Pastur [6]. Further research efforts were conducted to estimate the LSD of a product of two random matrices. Yin and Krishnaiah [7] proved the existence of LSD for the matrix sequence S n T n when the two matrix sequences S n and T n are standard Wishart matrices and positive definite matrices, respectively. In particular, Bai et al. [8] obtained a display representation of the LSD of the matrix sequence S n T n when the two matrix sequences S n and T n are the sample covariance matrix and the Wigner matrix, respectively. To relax the independence of the entries of X i , Silverstein [9] considered the case of X i � Mx i , where M is a nonnegative definite matrix and x i consists of independent and identically distribution (i.i.d.) entries, and the sample covariance matrices take the form of Random matrices of the form B n were extensively investigated by many researchers, including Yin [10], Silverstein [9], Bai and Zhou [11], Jin et al. [12], Zhang et al. [13] and Yao [14]. Silverstein's important work [9] aimed to relax the independence structure between X i coordinates and considered random vectors of the form Mx i , assuming that the spectral norm of the matrix sequence MM ⊤ is bounded and that the ESD of the matrix sequence MM T is convergent. Silverstein proved that the LSD of the sample covariance matrix B n exists, given by the LSD of equation representation of the Stieltjes transformation.
is paper investigates the LSD of large-dimensional sample covariance matrices generated by the PAR model, and the sample covariance matrices take the form of B n . A similar ARMA-type processes problem has been considered by Bai and Zhou [11], Jin et al. [12], Yao [14], etc. It follows that, in the ARMA-type processes, the covariance matrices MM ⊤ equals to the Toeplitz matrix, according to the fundamental eigenvalue distribution theorem for Toeplitz-type matrices [15], the ESD of MM ⊤ weakly converges to the nonrandom distribution H. For some situations, the ESD of MM ⊤ have explicit form; then, the LSD for the sample covariance matrix B n can be solved explicitly. For the random matrix generated by the first-order vector autoregressive (VAR(1)) model and the first-order vector moving average (VMA(1)) model, Jin et al. [12] and Bai and Zhou [11] obtained a display representation of this LSD. However, in our first-order variable-coefficient PAR process, we consider the process be interrupted by certain factors that the odd terms and the even terms of the coefficient α v converge to α and β, respectively, and that |α| ∈ (0, 1) and |β| ∈ (0, 1). e mixing matrix Q is a lower triangular matrix whose main diagonal elements are all 1 and subdiagonal elements differ from each other. When α � β, the limiting form of the matrices QQ ⊤ are Toeplitz matrices, and the eigenvalues of Toeplitz matrices have been proved by Gregory [16]; when α ≠ β, the ESD of QQ ⊤ has no explicit form, see [17,18], and the structure of the limiting matrices Q − 1 (Q − 1 ) ⊤ are very complex. Our main contributions in this paper include the following. (i) For the firstorder variable-coefficient PAR model, the explicit forms of limiting spectral moments for Q − 1 (Q − 1 ) ⊤ are given; through verifying the Carleman condition, we derive the explicit spectral moments of (1/n) n i�1 X i X ⊤ i . (ii) A framework of the maximum entropy-based method is provided for estimate the LSD. e paper is organized as follows. In Section 2, we employ the moment method to deduce the spectral density of sample covariance matrices. In Section 3, we use the maximum entropy method to find the approximate solution and analyze the convergence of the maximum entropy solution. We simulate the PAR model to obtain the periodic sample covariance matrices, and the experiment shows that the maximum entropy spectral density function curve can match histograms of the eigenvalues of the covariance matrices very well.

Methodology and Main Results
In this section, we first derive the explicit spectral moments form of Q − 1 (Q − 1 ) ⊤ ; subsequently, we verify the spectral moments statistics satisfying the Carleman condition, and then, we employ the moment method to deduce the spectral density of sample covariance.
We consider first-order PAR model: en, the first-order PAR model can be defined as where ⇒ denotes the equation on the left can push the equation on the right. From equation (3), Q is a lower triangular matrix whose main diagonal elements are all 1, and therefore, the matrix Q is invertible. Hence, Theorem 1. Let X ip+v be generated by a first-order PAR model in equation (2), and the odd terms and the even terms of the coefficient α v converge to α and β, respectively, and |α| ∈ (0, 1), |β| ∈ (0, 1). e error ε ip+v is independent random variables satisfying for any δ, ) ⟶ 0; furthermore, we assume that p/n ⟶ c ∈ (0, ∞); then, the spectral moments of sample covariance matrix ( where μ m is the limit of spectral moments of Q − 1 (Q − 1 ) ⊤ , which have an explicit form: Proof. e proof is divided into two steps. We first show the existence of the limit of spectral moments which is defined as ; then, we verify the Carleman condition. From equation (3), the inverse of matrix Q can be written as where a ij can be written as Let en, Furthermore, we obtain that Applying the trace operator, we have Based on the assumption that and therefore, Let en, e following section wishes to obtain a simplified form of the function h(x). From equations (10) and (16), we have en, h(x) can be rewritten as Let Journal of Mathematics can have a simplified representation as By the derivation operations, we obtain that By partially replacing x in equation (24) with y, we define a new function Define by which so en, We use x to take place of y and obtain that Next, we verify the Carleman condition en, applying the equality C n 2n � n j�0 (C j n ) 2 and |α| ∈ (0, 1), |β| ∈ (0, 1), we obtain From Stirling's formula n! � (1 + o(1)) ��� � 2πn √ n n e − n , we get n! ≥ n n e − n . en, 2)e/(2k − 1)) 2k− 1 ; applying this inequality, (μ 2k ) − (1/2k) has a finite lower bound: With the moment convergence theorem, the spectral distribution of Q − 1 (Q − 1 ) ⊤ tends to a nonrandom probability distribution H. Let S � (1/n) n i�1 ε i (ε i ) ⊤ and p/n ⟶ c ∈ (0, ∞); by the important eorem 2.10 of Bai [19], the ESD of (1/n) n i�1 XX ⊤ tends to a nonrandom limit distribution in probability (or almost surely). e spectral moments of (1/n) n i�1 XX ⊤ satisfy the following equation of Yin [10]: From the moment convergence theorem, we can obtain that the ESD of (1/n) n k�1 XX ⊤ converges with probability close to 1 to a nonrandom distribution, determined by the spectral moment statistic. □ Remark 1. By the proof of eorem 1, the limiting spectral moments of Q − 1 (Q − 1 ) ⊤ depend only on the limiting coefficient α and β. When α � β, the limiting form of the matrices QQ ⊤ are Toeplitz matrices, and the eigenvalues of Toeplitz matrices have been proved, see [16]. When α ≠ β, the limiting matrices of QQ ⊤ are tridiagonal matrices and take the form like the following: · · · · · · · · · · · · · · · · · · · · · · · · (37) e eigenvalues of this limiting matrix have no explicit form, see [18] and [17]. e limiting form of the matrices Q − 1 (Q − 1 ) ⊤ are very complex, and we obtain the explicit spectral moments of Q − 1 (Q − 1 ) ⊤ .

Estimating Spectral Density Function with Maximum Entropy
Unfortunately, there are no explicit LSD forms of this sample covariance matrices B n . In this section, with the spectral moments of β((1/n) n i�1 XX ⊤ ) k , we employ the maximum entropy method to estimate the LSD. Mead and Papanicolaou [20] studied the maximum entropy method to solve the uncertainty problem. Granziol et al. [21] demonstrated the equivalence between the MaxEnt principle and the optimization of variables with constraints. Gao and Han [22] used the maximum entropy method to solve nonlinear expectation. Every possible event has a probability of occurring, information entropy is related to all possibilities, and a system with the highest information entropy is most likely, when all known information, taken into account. In mathematics, the variable is generally estimated by maximising the Shannon entropy of the information of the moments of the random variable of the density function: where Ω is the support of the distribution. In this paper, the sample covariance matrices Q − 1 ((1/n) n i�1 ε i (ε i ) ⊤ )(Q − 1 ) ⊤ is nonnegative definite, so the estimated limiting distribution should be supported on Ω � R + ∪ 0 { }. Given the first N + 1 moments β 0 , β 1 , . . . , β N , the idea of determining the density function of a probability distribution based on the principle of maximum entropy is to find the density function p(x) such that where β 0 � 1. Lagrange multiplier is a method for finding the extremes of a multivariate function under a set of constraints. With the help of the Lagrange multiplier method, the entropy function is defined as When the first-order derivative of the function H with respect to λ k and p(x) are equal to zero, the constraint defined in equation (39) is returned; then, the first-order derivative of the function do not provide any additional information.
e second of these derivatives, however, evaluate to Journal of Mathematics 5 e general solution of equation (41) is where we have set (λ 0 − 1) ⟶ λ 0 . We obtain the maximum entropy solution by solving for the Lagrange multiplier λ k . To do this, we must solve the following nonlinear equation: When N ≤ 2, we can obtain a display representation of the Lagrange multiplier. Taking N � 2 as an example, from equation (36), we have Based on equation (40), we obtain λ 0 , λ 1 , λ 2 as en, When N ≥ 3, we cannot obtain a display solution for the Lagrange multiplier λ k , but at this point, the numerical solution for λ k can be calculated using numerical optimization algorithms. From equation (40), the analytic expressions of the gradient is If we denote the maximum entropy solution obtained by the method based on moments as then the Hessian matrix is given by μ k is the kth moment of the simulated distribution, and the Newton iteration is done by searching λ k until the termination criterion |μ k − μ k | < ε holds. We obtain the spectral density p(x) through Algorithm 1.
We can obtain that the maximum entropy solution p N (x) is weakly convergent, i.e., In our simulations, we use the Newton method from the Python-based SciPy package to solve for minimisation |μ k − μ k |. It computes the gradient within a ending accuracy ε as well as the Hessian matrix. To give the Hessian matrix a faster rate of convergence, we symmetrize it and add a disturbance of intensity η � 10 − 4 along the diagonal. In the simulation of the algorithm, we observe that the algorithm is not too sensitive to these choices.
From equations (31) and (36), we obtain the four limiting spectral moments (1/n) n i�1 X i (X i ) ⊤ of the sample covariance matrices as follows: e authors of [23,24] state that, as the number of moments increases (especially when n ≥ 5), the maximum entropy method becomes sensitive to numerical differences between the order moments, the gradient, and the state of the Hessian matrix, as well as improper arithmetic accuracy becomes very sensitive and can be improved by introducing different basis functions, such as a drift with Chebyshev polynomial basis functions with drift to overcome these effects. In this section, to simplify the operations, we estimate the maximum entropy density function in terms of 4 th -order spectral moments. e value of (d; n; a; b) determines the value of each spectral moment. Figures 1-6 show histograms of eigenvalues and maximum entropy spectral density curves (MaxEntPDF) for different values of (d; n; a; b). From these plots, it can be seen that the maximum entropy spectral density function curve can match histograms of the eigenvalues of the covariance matrices very well. Input: β � (β 0 , . . . , β N ), Ending accuracy ε, Diitter variance in Hessian η � 10 − 4 ; Output: Coefficients λ i ; initial λ 0 � (− ln( �� � 2π √ ), 0, . . . , 0), k � 0; repeat Compute gradient g � (g 0 , . . . , g N ): g j � β j − Ω x j exp( N i�0 λ ki x i )dx; Compute Hessian G; Direction of Newton's law: d k � − (G + ηI) − 1 · g; Next iteration point: λ k+1 � λ k + d k ; k � k+1; until ‖β − μ‖ 2 < ε. ALGORITHM 1: e proposed algorithm.

Conclusion
In this paper, we investigate the LSD of large-dimensional sample covariance matrix generated by the first-order variable-coefficient PAR process. We consider the process be interrupted by certain factors that the odd terms and the even terms of the coefficient α v converge to α and β, respectively, and that |α| ∈ (0, 1) and |β| ∈ (0, 1). e main contributions in this paper include the following. (i) For the first-order variable-coefficient PAR model, the explicit form of limiting spectral moments for Q − 1 (Q − 1 ) ⊤ is given; through verifying the Carleman condition, we derive the explicit spectral moments of (1/n) n i�1 X i X ⊤ i . (ii) A framework of the maximum entropy-based method is provided for estimate the LSD. We have some ideas for the next step of the research. On the one hand, when generalizing the convergence of the coefficients of the PAR model to a more general structure, how about obtaining an expression for the spectral moments of the limiting covariance matrix? On the other hand, it is of importance to extend the method presented in this paper to PAR with larger order.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that they have no conflicts of interest.