Efficient Techniques for Solving the Periodic Projected Lyapunov Equations and Model Reduction of Periodic Systems

We have presented the efficient techniques for the solutions of large-scale sparse projected periodic discrete-time Lyapunov equations in lifted form. These types of problems arise in model reduction and state feedback problems of periodic descriptor systems. Two most popular techniques to solve such Lyapunov equations iteratively are the low-rank alternating direction implicit (LR-ADI) method and the low-rank Smith method. The main contribution of this paper is to update the LR-ADI method by exploiting the ideas of the adaptive shift parameters computation and the efficient handling of complex shift parameters. These approaches efficiently reduce the computational cost with respect to time and memory. We also apply these iterative Lyapunov solvers in balanced truncation model reduction of periodic discrete-time descriptor systems. We illustrate numerical results to show the performance and accuracy of the proposed methods.

Model reduction means to replace the given mathematical model ( 1) by a much smaller model which preserves certain crucial properties of the original model, such as stability and passivity, and it is then amenable for simulation and analysis.Analysis and reduced-order modeling of system (1) may require solving large-scale sparse projected periodic discrete-time Lyapunov equations.Model reduction procedures of such periodic systems using direct solution approaches have been considered in [6][7][8][9][10].Recently, attention has been devoted to iterative solutions of large-scale sparse Lyapunov equations using the alternating directions implicit (ADI) method [11,12], the Smith method [12][13][14], and Krylov subspace methods [15,16].Generalization of all these methods has also been considered in [17,18].
However, low-rank iterative methods and their application in model reduction of such periodic descriptor systems have been considered in [19,20].The drawbacks of the ADI method proposed that there are the suboptimal shifts computations and the slow convergence of the iterative solutions to the exact solutions.In this paper, we proposed the adaptive shift parameters [21] computation instead of suboptimal parameters and explored the efficient handing of the complex shift [22] parameters.These two proposed schemes reduce the computational cost and fasten the convergence of the iterative solutions to exact solutions.Numerical results are discussed to show the efficiency of the techniques.
The paper outline is as follows.In Section 2, we briefly review the analogous time-invariant representation of the periodic systems, known as lifted representation of periodic systems.We discuss the cyclic lifted representation of periodic descriptor system (1).We reviewed the causal-noncausal decomposition, the periodic projected Lyapunov equations [8], and their corresponding lifted forms [7].We also review some important concepts of the discrete-time descriptor system in linear time-invariant (LTI) form and the balanced truncation (BT) model reduction in LTI case.We also discuss the basics of the generalized ADI method in this section.The main idea behind this is that the concepts in the LTI structures will help more precisely to understand their corresponding periodic interpretations.In Section 3, we discuss the efficient solution of the causal lifted Lyapunov equations using the adaptive shifts based LR-ADI method.We also reviewed the Smith method from [20] for noncausal lifted Lyapunov equations in Section 4. We consider a balanced truncation model reduction method for periodic descriptor systems in Section 5. Section 6 contains numerical results that illustrate the efficiencies of the described iterative methods in model reduction of periodic descriptor system.Section 7 deficits the conclusion and remarks.

Preliminaries
In this section, we review the analogous time-invariant representation of the periodic systems, known as lifted representation.We also introduce some notations and briefly review the fundamental concept of balanced truncation model reduction for LTI discrete-time systems.We, afterward, generalize these concepts and results in the periodic setting.

Cyclic Lifted Representation of Periodic Systems.
Analysis and modeling of periodic discrete-systems are often described by an analogous time-invariant representation of the periodic systems, known as lifted representation [1,2,6].Here we consider the cyclic lifted representation [4] of periodic descriptor system (1), given by where The descriptor vector, system input, and output of (2) are related to those of (1) via ] , respectively.We can define the important concepts and dynamics of the periodic systems, such as regularity and asymptotic stability, by using their corresponding lifted forms.

Balanced Truncation for LTI Discrete-Time Systems.
In order to understand the BT model reduction method for the time-varying discrete-time systems, we briefly review the idea for the time-invariant case.Consider a LTI discrete-time system: in which  ∈ R   ×  is nonsingular, and  ∈ R   ×  ,  ∈ R   ×  , and  ∈ R   ×  .Assume that the original system is asymptotically stable; that is, all finite eigenvalues of the system pencil lie inside the unite circle; then the controllability Gramian  ∈ R   ×  and the observability Gramian  ∈ R   ×  can be computed by solving the discretetime Lyapunov equations (see, e.g., [24]): Equivalently, to compute the controllability and observability Gramians we can solve the continuous-time Lyapunov equations where  −  š ( − ) − ( + ) is the Cayleytransformed pencil [25], and  = √ 2 , and  = √ 2 .
Note that the solutions of ( 12) and ( 13) are identical to the solutions of ( 14) and ( 15), respectively [20,25].It is true that the Cayley transformation can destroy the system's sparsity pattern where it needs to be restored.But, using proper permutations and efficient computation, one can recover this difficulty.
Since both the Gramians are symmetric and positive semidefinite, they have symmetric Gramians factors The main drawback of BT based model order reduction (MOR) techniques is the computation of the solutions of the Lyapunov equations that are used in computing the Gramian factors.During the recent decades several approaches have been developed that allow exploiting the fact that often all coefficient matrices are highly sparse and the numbers of inputs and outputs are very small compared to the number of DoFs.Then usually the Gramians  and  can be expressed in low-rank factored forms; that is,  and  are thin rectangular matrices.Therefore instead of computing the full Gramian factors it is quite reasonable to compute the thin rectangular matrices  and .

Low-Rank ADI for LTI Discrete-Time Systems.
One popular iterative method for computing the low-rank  and  is the generalized low-rank Cholesky factor-alternating directions implicit (GLR-ADI) iteration [11,26,27].In the following we briefly introduce the GLR-ADI methods to solve continuoustime algebraic Lyapunov equations ( 14) and (15).For model reduction of system (1) we have to solve such a kind of Lyapunov equations.
A set of optimal ADI shift parameters or simply shift parameters {  }  =1 ⊂ C − are necessary for fast convergence of the GLR-ADI iterations.We will discuss the shift parameter selection criterion later in this section.In these iterations we also see that if the maximum number of iterations  max is greater than the number of shifts , then the shift parameters are used in a cyclic way.
In the GLR-ADI iterations all of the input matrices , , and  are real, due to the complex shift parameters in each iteration step, the updated   store complex data, which increases the overall complexity and memory requirements of the method.Moreover, in the balancing based methods using these complex Gramian factors yields complex reduced systems by performing some complex arithmetic operations.This problem is resolved in [22].In this regard, the important assumption is that the selected ADI shift parameters should be proper.Definition 3. The ADI shift parameters {  }  =1 ⊂ C − are called proper if   and  +1 are complex conjugates of each other when one of them is complex.
In [22] it is shown that at the ( + 1)-st iteration of the GLR-ADI iterations,  +1 can be computed by where  = Re(  )/Im(  ).This identity states that, for  +1 =   ,  +1 can be computed explicitly from   , which releases us from solving a shifted linear system with  +   .This idea also results in the following theorem.
Theorem 4. Let us assume a set of proper ADI shift parameters.For a pair of complex conjugate shifts {  ,  +1 fl   }, the two subsequent block iterates   and  +1 of GLR-ADI iterations satisfy [22] [ This theorem reveals that for a pair of complex conjugate shifts at any iteration step in the GLR-ADI iteration   can be updated by A version of the GLR-ADI algorithm is summarized in [29, Algorithm 5], which computes low-rank real Gramian factors.Additionally, the GLR-ADI iterations can be stopped whenever the norm of the ADI-residual becomes very small.But computing ‖F(  )‖ in Frobeniusnorm or 2-norm in each iteration step is an expensive task, since in each iteration the resulting residual ( 21) is an   ×   matrix.Recently, in [30] the authors show that, in the th iteration, the ADI-residual in ( 21) can be represented as with and the   in the GLR-ADI iterations can be expressed as [30]   = ( +   ) The authors in [30] also show that the ADI-residual factor defined in (23) can be computed by the expression In the case of real setting when we consider  +1 fl   , one must compute where  is defined in (18).The rank of   is at most   , that is, the number of columns of .Therefore, the computation of the Frobenius-norm or 2-norm of ‖     ‖ = ‖     ‖ in each iteration is extremely cheap.Applying these strategies (computation of real Gramian factors and low-rank residual based stopping techniques), the updated GLR-ADI will be briefly discussed in Section 3.

Efficient Solutions of Causal LPDALEs Using Low-Rank ADI Methods
The basic ADI method for linear systems has been proposed in [31] and then used in [11,12,17,[32][33][34] for solving (projected) continuous-time and discrete-time Lyapunov equations.Consider the LPDALE (9a), where the matrix E = diag( 0 , . . .,  −1 ) is singular.For such equations both the ADI and Smith iterations fail to converge [20].This problem is circumvented by considering a generalized Cayley transformation [25]: which transforms the LPDALE (9a) to an equivalent projected continuous-time algebraic Lyapunov equation (PCALE): Here E − A = (A − E) − (A + E) is the Cayley-transformed pencil.In this transformation the finite stable eigenvalues of E − A are mapped to the finite stable eigenvalues of E − A, and the infinite eigenvalue of E − A is mapped to  = 1.
More details of that transformation can be found in [20,25].
The solutions of PCALE ( 28) and (9a) are unique and they are solvable for every BB  [20].

Efficient Computation of Low-Rank Gramian Factors.
We present the updated version of low-rank ADI methods for Lyapunov equation (28), which is also the solution of (9a).An updated algorithm for the low-rank ADI method of generalized state space systems can be found in [29].Here the idea is generalized for the periodic descriptor system.Applying ADI method the solution G cr of Lyapunov equation ( 28) can be obtained by performing the following iteration steps, beginning with G cr 0 = 0 and  = 1: where   ∈ C − , with  = 1, 2, . . .,  max , is a shift parameter and B = √ 2P  B. Steps (29a) and (29b) can be written in a single step where The error bound in iteration  is given by (see, e.g., [35]) Since G cr 0 = 0, this leads to In th iteration the residual can be computed by where W  = ∏  =1 A   B. We observe that the computed G cr  in ( 30) can be decomposed as where Z  ∈ R ×  and   ≪ .Since the number of inputs is very small compared to the dimension of the system, the right hand side of (9a) is a low-rank matrix.Therefore often this decomposition is possible, and Z  can be computed by following iteration steps: with  = 2, 3, . . .,  max , and  max is maximum number of iterations.In each iteration in (35b) the number of columns of Z  is increased by the number of columns Z −1 .Therefore after few steps the number of columns of Z  grows dramatically.In order to resolve this drawback a further optimization occurs, where the number of columns to modify Z  is kept constant by using the clever reordering of the ADI-parameters.This updated version of GLR-ADI [36] is given below: Unfortunately, the GLR-ADI iteration does not preserve the block diagonal structure at every iteration step in Algorithm 1.But G cr  = Z  Z   has almost block diagonal form since it approximates the solution of the LPDALE (9a) given by G cr = diag( cr 1 , . . .,  cr −1 ,  cr 0 ), where  cr  are the periodic solutions of (5).We then consider the block partitioning with   ∈ R   ×  , and the causal reachability Gramian  cr  can be approximated by the matrices      for  = 0, 1, . . ., −1.We compute low-rank Gramian factor Z, so that ZZ  ≈ G cr by following the steps (36a)-(36c).We summarize the resulting iteration using column compression in Algorithm 1.It should be noted that in Algorithm 1 the computation of the matrices E and A can be avoided by rewriting the GLR-ADI iteration in terms of the original matrices E and A [20].We should also note that we do not compute the inverses of the matrices at step (2) of Algorithm 1 explicitly.We represent the matrices E and A with the original sparse matrices with E and A and use the sparse inversion technique in step ( 2) [20].
The iteration can be stopped in the th step if the residual norm, that is, ‖Res  ‖, where Res  is defined in (33), is sufficiently small.If we can compute W  as a low-rank factor then computing ‖Res‖ is very cheap.Here, we discuss how to compute the low-rank residual-factor from each iteration in the GLRCF-ADI algorithm.Following the procedure as shown in Section 2.4, iterate V  in (36b) can also be written as Also, (39)

Selection of the ADI Shift Parameters.
The convergence rate of the low-rank ADI algorithms discussed above depends on a set of ADI shift parameters.The ADI shift parameters were originally introduced by Wachspress in [37] to solve Lyapunov equations using the ADI methods.The author shows that a set of optimal ADI shift parameters {  }  =1 for system (2) can be computed by solving the so-called ADI minmax problem [37,38]: where Sp(E −1 A) is the spectrum of the matrix E −1 A.
For a large-scale system, determining the entire spectrum of (A, E) is almost impossible.Therefore, in the literature several techniques are proposed; see, for example, [26,36,39,40] to solve the min-max problem (40) using a much smaller part of the spectrum.Currently, one such commonly used technique is Penzl's heuristic approach introduced in [26], where  + Ritz values (see, e.g., [41]) and  − ( + ,  − ≪ ) reciprocal Ritz values with respect to E −1 A and A −1 E, respectively, are employed.A complete procedure of the heuristic approach can be found in [20].However, for largescale descriptor systems, computing the Ritz values with respect to E −1 A is a challenging task since E is then not invertible.
Another ADI shift selection criterion that we emphasize here is the adaptive approach which is introduced in [21].The performances of this adaptive shift selection approach for a large-scale system are also investigated in [29,42,43].There the authors show that the approach is superior to the heuristic approach, especially for the (generalized) descriptor systems.To initialize the shifts, we first compute the eigenvalues of the projected pencil where U is formed by the basis of range of any   ×  ( ≪   ) random matrix.Then select some desired number of optimal shifts by solving the ADI min-max problem like in the heuristic procedure.Then whenever all the initial shifts have been used, the pencil is projected to the span of the current   (see, e.g., Algorithm 1 of [21]) and computes some optimal shifts from the eigenvalues of the projected pencil and uses them as the set of next shift parameters.This approach is repeated while the algorithm has not converged to the given tolerance.

Smith Method for Noncausal LPDALEs
Consider the LPDALE (9b).For nonsingular A, this equation is equivalent to the LPDALE: Note that if  is the index of the system [20], then the approximate solution of the noncausal LPDALE (9b) can be obtained by the Smith iterations [14]: Therefore, the Cholesky factor   of the solution G ncr =      of (42) and also of the LPDALE (9b) takes the form Thus, the solution of (9b) can be obtained with few computations for systems of low index.The solution of the noncausal LPDALE (9b) is briefly discussed in [20].Similar to the causal case, we consider the block partitioning such that the noncausal reachability Gramians of system (1) can be computed in factored form  ncr  = X X  for  = 0, 1, . . .,  − 1.

Remark 5. The causal and noncausal observability Gramians 𝐺 co
and  nco

𝑘
of system (1) and the corresponding the lowrank Cholesky factors   and Ỹ , where  co  ≈      and  nco  = Ỹ Ỹ  , can also be determined from the corresponding LPDALEs that are dual to the LPDALEs (9a) and (9b); see [7] for details.

Model Reduction of Periodic Systems
For periodic descriptor system (1), we construct a reducedorder model of dimension  of the form where Ẽ ∈ R  +1 × +1 , Ã ∈ R  +1 ×  , B ∈ R  +1 ×  , C ∈ R   ×  are -periodic matrices, ∑ −1 =0   = ∑ −1 =0   = , and  ≪ .Apart from having a much smaller state space dimension, the reduced-order model also preserves important physical properties of the original system such as regularity and stability and the small approximation error.Balanced truncation for discrete-time periodic descriptor system has been considered in [7,20].Some important concepts related to balanced truncation of discrete-time periodic descriptor systems has been presented in [8].
Consider {(  ,   )} −1 =0 is periodic stable, and the Cholesky factors of the causal and noncausal Gramians satisfy Then the causal and noncausal Hankel singular values  , and  , of periodic descriptor system (1) are defined as respectively, where   (⋅) denotes the singular values of the corresponding product matrices.For a balanced system, one truncates the states related to the small causal Hankel singular values because they do not change system properties essentially.Unfortunately, truncation of small nonzero noncausal Hankel singular values may lead to unstable reduction [7].Hence, we compute the singular value decompositions of the product matrices where [ ,1 , where the projection matrices  , and  , have the form with be the transfer function of lifted system (2) and H() be the transfer function of the corresponding reduced-order lifted system formed from the reduced-order matrices in (51).Then the H ∞ -norm error bound for the reduced-order system is given by where Σ ,2 ,  = 0, 1, . . .,  − 1, contains the truncated causal Hankel singular values from (49).

Numerical Experiments
In this section, we present some numerical results.We consider an artificial periodic discrete-time descriptor system which is reformed from its original model in [8, Example 1].
The reformulation of the model is described in [20].Here all the computations are carried out using MATLAB5 R2012b on an Intel CORE i5 processor with a 2.90-GHz clock and 16 GB of RAM.
To show the comparisons of GLR-ADI (i.e., Algorithm 2 in [20]) and our UGLR-ADI (i.e., Algorithm 1), we compute the low-rank Gramian factors by solving the LPDALEs applying both the algorithms.We also investigate the performances of both the heuristic and adaptive shifts to implement these algorithms.For implementing the GLR-ADI iteration we chose 40 optimal heuristic shifts out of 50 large and 40 small magnitude Ritz values.To select 40 heuristic shifts it takes 3.98 seconds of CPU time.In case of the adaptive shifts (for implementing UGLR-ADI method), in each cycle 40 proper shift parameters are selected following the procedure discussed above.We require only 0.03 seconds of CPU time to compute this number of adaptive shift parameters.Note that for computing the initial shifts first we project the pencil (A−E) onto the column space of an   ×100 random matrix.
Figures 1 and 2, respectively, show the rate of convergence and the computational time of the GLR-ADI and UGLR-ADI methods.In both cases, we see that UGLR-ADI method performs better than the GLR-ADI method.We compute the reduced-order model of the periodic descriptor system by using the low-rank Gramian factors obtained from UGLR-ADI method.The reformed periodic descriptor system has

Conclusions
In this article we have suggested the adaptive shifts based ADI method and reviewed the Smith method for computing the low-rank factors of the periodic Gramians for periodic discrete-time descriptor systems.Later, these factors are used in a balancing based model reduction technique to find a reduced-order model for the periodic discrete-time descriptor system.The proposed techniques save memory and the computational time to generate a reduced-order model.The capability and efficiency of the methods have been reflected in the numerical results.

Figure 1 :
Figure 1: Comparisons of the old algorithm (GLR-ADI) and new algorithm (UGLR-ADI) in terms of convergence rate in computing the low-rank Gramian factors.

Figure 2 :
Figure 2: Comparisons of the old algorithm (GLR-ADI) and new algorithm (UGLR-ADI) in terms of required time in computing the lowrank Gramian factors.

Figure 3 :
Figure 3: Frequency responses and error bounds of the original and the reduced-order model.
Figure 3(a)  shows the frequency responses of the original and reduced models match nicely.The estimated error is below the error bound which is reflected in Figure3(b).