Conditional Nonlinear Optimal Perturbation of a Coupled Lorenz Model

The conditional nonlinear optimal perturbation (CNOP) technique is a useful tool for studying the limits of predictability in numerical weather forecasting and climate predictions.TheCNOP is the optimal combinedmode of the initial andmodel parameter perturbations that induce the largest departure from a given reference state.The CNOP has two special cases: the CNOP-I is linked to initial perturbations and has the largest nonlinear evolution at the time of prediction, while the other case, CNOP-P, is related to the parameter perturbations that cause the largest departure from a given reference state at a given future time. Solving the CNOPs of a numerical model is a mathematical problem. In this paper, we calculate the CNOP, CNOP-I, and CNOP-P of a coupled Lorenz model and study the properties of these CNOPs.We find that the CNOP, CNOP-I, and CNOP-P always locate the boundary of their respective constraints. This property is also demonstrated analytically for the model whose solutions depend continuously on the initial and parameter perturbations, which provides a theoretical basis for testing the accountability of the numerically computed CNOPs. In addition, we analyze the features of the CNOPs for the coupled Lorenz model and explain their structures.


Introduction
Error growth dynamics is one of the fundamental problems in the study of numerical weather forecasting and climate prediction.The prediction errors are generally caused by initial and model errors.To study the role of initial and model errors in yielding prediction errors, Lorenz [1] classified two types of predictability problems: one is related to the initial error, assuming a perfect model, and is referred to as the first type of predictability; the other is associated with model errors, assuming a perfect initial field, and consists of the second type of predictability.The former has been largely investigated, and many theories and methods have been proposed or introduced [2][3][4] in which a singular vector (SV; [2,5]) is often used.However, SVs are the result of a linearized model and do not consider the effect of nonlinearity.Considering the limitation of the linear theory of SV, Mu et al. [4] proposed the approach of conditional nonlinear optimal perturbation (CNOP) to search for the optimal initial perturbation (denoted by "CNOP-I") in a given constraint.The CNOP-I considers the effect of nonlinearity and is more effective than the SV at revealing the effect of nonlinearity.
CNOP-I has been applied to predictability studies for weather and climate [6][7][8] and has played an important role in guiding scientists to develop and improve numerical models and even to propose innovative ideas to increase the forecasting accuracy of weather and climate models [9].
The effect of model error on predictability is related to the second type of predictability [1].One important aspect in this field is the effect of the uncertainties in model parameters on predictability [10].In numerical models, there are generally several empirical parameters, and they may simultaneously have uncertainties.In predictability studies, one is required to find the patterns of the parameter perturbations that have the largest effect on predictability.Furthermore, the initial and model parameters' errors generally coexist in predictions.In this situation, one expects to know which combined mode of initial and parameter errors yields the largest effect on predictions.For this reason, Mu et al. [11] extended the CNOP-I to study parameter uncertainties and proposed the CNOP, which consists of the optimal combined mode of the initial and parametric perturbations.The CNOP induces the largest departure from a given reference state, which can be used to estimate the limit of prediction errors that is caused 2 Mathematical Problems in Engineering by initial and model parameter errors [11].If we only consider the effect of the initial perturbation, then the CNOP is only related to the initial perturbation, as is the CNOP-I.If we only consider the parametric perturbation, the CNOP is merely associated with the parametric perturbation and acts as the parametric perturbation that causes the largest departure from the reference state.In this case, the CNOP is denoted as CNOP-P.
To use the CNOP approach to study the limits of predictability, the CNOPs of models must be solved, which can result from a mathematical problem.CNOPs are related to several optimization problems.In this paper, we will solve the CNOP, CNOP-I, and CNOP-P of a coupled Lorenz model and study the properties of the CNOPs.In the following section, the CNOP approach is reviewed.The CNOPs of a coupled Lorenz model are explored in Section 3. Finally, a summary is presented in Section 4.

Conditional Nonlinear Optimal Perturbation
In this section, we follow Mu et al. [11] and Duan and Zhang [10] to review the CNOP approach.We assume the following evolution equations for the state vector, : where w(x, ) = ( 1 (x, ),  2 (x, ), . . .,   (x, )) and w 0 is its initial state.(x, ) ∈ Ω × [0, ], Ω is a domain in   ,  < +∞, x = ( 1 ,  2 , . . .,   ), and  is the time.Furthermore, p = ( 1 ,  2 , . . .,   ) is a model parameter (vector), and  is a nonlinear operator.Assuming that the dynamic system equations (1) and the initial state are known exactly, the future state can be determined by integrating (1) with the appropriate initial condition.The solution to (1) for the state vector w at time  is Here,   () is the propagator of (1) with the parameter vector  and, as described by (2), "propagates" the initial value to the time  in the future.
Let () and () + () be solutions to (2) with initial values  0 and  0 +  0 , respectively, where  0 is the initial perturbation of a time-dependent state () (hereafter the "reference state"), which describes the nonlinear evolution of this initial perturbation.Assume that a parameter perturbation   is superimposed on a reference parameter : where   () describes the departure from the reference state (), which is caused by   .Now, we explore the situation with both initial and parameter perturbations in () is the departure from the reference state (), which is caused by the combined mode ( 0 ;   ) of the initial perturbation  0 and the parameter perturbation   .
A nonlinear optimization problem is defined in where 0 ∈   , and   ∈   are, respectively, the constraining conditions of the initial and parameter perturbations, where the constraining conditions constrain the amplitude of the initial and parameter perturbations.By solving (6), one can obtain the optimal combined mode of the initial and parameter perturbations ( 0 ;   ) that, for the given constraint, induces the largest departure from the reference state () at time .This combined mode is the CNOP.When we consider only the initial perturbation or assume that the constraint of the parameter perturbation   = 0, then (6) becomes (8): The initial perturbation   0 that satisfies (8) is the CNOP-I, which has the largest nonlinear evolution at time .It is clear that the CNOP-I is a special case of CNOP.Another special case of the CNOP is the CNOP-P.In fact, in (6), we neglect the initial perturbation, that is, the constraint of the initial perturbation is  0 = 0, and we obtain in which    is the CNOP-P.CNOP-I, CNOP-P, and CNOP are generally vectors.Their magnitudes are constrained by the constraint conditions  0 ∈   ,   ∈   , and the two of  0 ∈   ,   ∈   , respectively (see (6), (8), and ( 9)).The constraint conditions and their boundaries are generally determined by the physical problems of the investigation.There are a large number of combinations of different components of initial perturbations or/and parametric perturbation vectors that satisfy the constraint conditions.The goal of the CNOP is to determine the combination that causes the largest departure from the reference state.Therefore, in predictability studies, despite the determination of the boundaries of the constraint conditions, the CNOP must still be calculated to explore the initial errors (or parametric errors, or the combination of the two) that cause the largest prediction errors at the time of prediction.Initial and parametric perturbations represent the initial and parametric errors in weather or climate models, respectively.Initial errors are often referred to as the initial observational errors.Observational errors are usually caused by observational instruments and observational methods and, consequently, have a finite amplitude.The amplitude of observational errors can be regarded as the amplitude of the initial errors.Therefore, initial errors are finite, and the boundaries of the constraint conditions are finite.Model parameters are used to describe the effect of certain physical processes, which, limited by our understanding and realization of physical processes, may be empirical and have uncertainties.Of course, the uncertainties in the model parameters should have finite amplitudes.Otherwise, the corresponding numerical models fail to describe the physical processes to an acceptable extent, which, as a result, cannot be regarded as forecasting models.Therefore, the boundary of the constraint conditions in (6) (i.e.,  0 ∈   or   ∈   ) should have a certain amplitude based on physics, which, mathematically, should be constrained in the neighborhood of reference states.
As mentioned in the introduction, CNOP-I is a natural generalization of LSV in a nonlinear field.Mu et al. [4] compared CNOP-I and LSV and demonstrated that, when CNOP-I is computed with a linearized model, CNOP-I is the LSV.In addition, Mu et al. [4] also demonstrated that when the CNOP-I is computed with a nonlinear model, a CNOP can be approximated by an LSV only when the initial perturbations are sufficiently small; otherwise, the CNOP cannot be approximated by the LSV (also see [12]).In this paper, we calculate not only the CNOP-I but also the CNOP-P and CNOP.The latter two are also related to parameter perturbations.When a nonlinear model is linearized, some parameters may be removed due to linearization of the nonlinear model.In particular, if we linearize the coupled Lorenz model that is adopted in this paper, the parameters  and  will be removed.Thus, we cannot consider the effect of the uncertainties of all of the parameters on the prediction results.Therefore, it is not reasonable to consider the CNOP-P and CNOP for a linearized model.To coordinate CNOP-I with CNOP-P and CNOP, we do not calculate the CNOP-I of the linearized coupled Lorenz model in this paper.

CNOP, CNOP-I, and CNOP-P of a Coupled Lorenz Model
In this section, we will illustrate the CNOP, CNOP-I and CNOP-P for a coupled Lorenz model (see [13]).The coupled Lorenz model is given in In this model, there are five parameters; in the numerical experiments, these parameters are set as  = 10,  = 0.12,  = 28,  = 8/3, and  = 0.3 (see [13]).In this paper, the model ( 10) is discretized by the fourth-order Runge-Kutta scheme, with a time step  = 0.01.It is known that the Lorenz model has three steady states [14]:  : (, , ) = (0, 0, 0) , (11) However, we note that these steady states in the Lorenz model may be unsteady in the coupled Lorenz model.That is, if these states are  0 and the propagation of ( 10) is   , then there exists the possibility that  0 ̸ =   ( 0 ) in the coupled Lorenz model.In fact, the steady states  1 and  2 in the Lorenz model will be unsteady in the coupled Lorenz model, whereas the steady state  in the Lorenz model is also a steady state in the coupled Lorenz model.We can regard these states as reference states to compute the CNOP, CNOP-I, and CNOP-P of the coupled Lorenz model. is a steady reference state, and  1 and  2 are unsteady in the coupled model, yielding two time-dependent reference states.For convenience, we call them Ref-, Ref- 1 , and Ref- 2 , respectively.
In the objective function  in (7), we take  0 as the three states of (, , ) and  as the five parameters (, , , , ) in the coupled Lorenz model to obtain their CNOP, CNOP-I, and CNOP-P.We signify the initial perturbations as  0 = ( 0 ,  0 ,  0 ) and the parameter perturbations as   = (, , , , ).Then, the objective function ( 0 ;   ) = √ () 2 + () 2 + () 2 and describes the departure from the reference state ((), (), ()) that is caused by the initial perturbation  0 and the parameter perturbation   .The conditions are used to constrain the magnitudes of the initial and parameter perturbations, respectively, where  and  are two given positive numbers.For the different values of  and , we obtain different magnitudes of constraints.
To compute the CNOP, CNOP-I, and CNOP-P, we need the information from the gradient of the objective function .Mu et al. [11] and Duan and Zhang [10] derived the formula for the objective function with respect to the initial and parametric perturbations.The gradient of the objective function  with respect to the initial and parametric perturbations can be obtained with the following formula: where  1 () and  2 () satisfy Equation ( 14) is the adjoint of the tangent linear model from (1).By integrating (14), we can obtain the gradient for the objective function ( 0 ;   ) with respect to  0 and   .With the gradient / 0 =  1 (0)(/  =  2 (0)), the CNOP-I (CNOP-P) can be computed using optimization solvers, such as the spectral projected gradient 2 (SPG2, [15]), sequential quadratic programming (SQP, [16]), and limitedmemory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS, [17]).For both gradients / 0 =  1 (0) and /  =  2 (0), one can calculate the CNOP of the initial perturbation and the parameter perturbation.
Equation (14) suggests that the gradient (/ 0 , /  ) of the objectivee function  in ( 7) can be obtained by integrating the adjoint model (14).With this gradient information, we can obtain not only the CNOP but also the CNOP-I and the CNOP-P using an optimization solver.
Next, we investigate the CNOPs that consist of the combined mode of the initial and parameter perturbations for the given reference states Ref-, Ref- 1 , and Ref- 2 , where the optimization time period  associated with CNOPs is equal to 50, with a time step  = 0.01 (see (10)).According to (14), we construct the tangent linear model for (10) and then the adjoint model.By integrating this adjoint model, we obtain the gradient for the objective function  with respect to the initial and parameter perturbations.Next, we use this gradient to compute the CNOP, CNOP-I, and CNOP-P of the coupled Lorenz model using the SQP algorithm [16].First, we evaluate the CNOPs for the reference state Ref- with the given objective function and the different amplitudes of constraints.It is demonstrated that there exist two CNOPs for the steady reference state Ref- for each constraint.For example, when the constraints are  = 0.1 and  = 0.05, both the perturbations ( 0 ;   ) = (−0.100,−0.100, 0.100, 0.500, 0.006, 1.400, −0.133, −0.015) and (0.100, 0.100, 0.100, 0.500, 0.006, 1.400, −0.133, −0.015) are the CNOPs of the reference state .It is easily seen that the difference between the perturbations only consists of the components  0 and  0 .These two CNOPs are symmetric about the -axis.This relationship is easily determined from (10).In fact, if ((), (), ()) is a solution to model (10), then (−(), −(), ()) is also a solution.For simplicity, we only list one of the two CNOPs in Table 1.From Table 1, it is evident that, for the different amplitudes of constraints, the CNOPs always locate the boundary of the constraint.
The reference state Ref- is steady and, together with the -axis, acts as a symmetric axis of the system described by the coupled Lorenz model, which induces its two symmetric CNOPs about the -axis.Note that the reference states Ref- With the corresponding objective function and constraint condition, we obtain the CNOPs for the reference states Ref- 1 and - 2 .The results demonstrate that these two reference states have one CNOP; furthermore, the components  0 and  0 between these two CNOPs are inverses of each other, while the other components remain the same (i.e., these two CNOPs are also symmetric about -axis).For simplicity, we only display the results for Ref- 1 in Table 2.
It is clear from Tables 1 and 2 that the CNOPs of these two reference states are different, which indicates that the CNOPs are dependent on the reference states.Furthermore, it is easily shown that the CNOPs of the reference states Ref- 1 and - 2 always locate the boundary of the constraint.
In numerical experiments, we also adopt other reference states to compute the CNOPs.Similar results are obtained.It is obvious that whether the reference states are steady or unsteady, the CNOPs of the coupled Lorenz model will always locate the boundary of the constraint.
CNOP consists of both initial and parameter perturbations.If one only perturbs the initial condition of a reference state and the remaining parameters are unperturbed, one can obtain a CNOP-I with (8).CNOP-I is one of the special cases of CNOP.For its computation, we still adopt the reference states Ref-, Ref- 1 , and Ref- 2 and consider which types of we calculate the CNOP-I for the given reference states.For the steady state , we obtain two symmetric patterns of CNOP-I due to the symmetry of the model (10).In Table 3, we list only one CNOP-I for the steady reference state .
From Table 3, it is apparent that the CNOP-I locates the boundary of the initial constraint.This property of CNOP-I can also be demonstrated by the CNOP-I of the unsteady reference states Ref- 1 and Ref- 2 .To illustrate this point, we list in Table 4 the CNOP-I of the Ref- 1 ; the Ref- 1 and Ref- 2 have one CNOP-I.Furthermore, due to the symmetry of (10), the CNOP-I of both Ref- 2 and Ref- 1 is symmetric about the -axis in phase space.For example, when the constraint  = 0.1, the CNOP-I related to  1 is ( 0 ,  0 ,  0 ) = (−0.1,−0.1, −0.1), while that of the Ref- 2 is (0.1, 0.1, −0.1).For simplicity, we only show the details for Ref- 1 .
In addition, we perform a comparison of the results in Tables 1-4.Although the CNOP-I is a special case of the CNOP, it may not have the same pattern as that of the initial perturbation in the CNOP pattern.That is, the CNOP cannot be a simple combination of CNOP-I and CNOP-P.This conclusion indicates that, in predictability studies, it is necessary to investigate the specific combination of initial and parameter errors that can cause the largest prediction error.To further address this result, we now study the CNOP-P for the given reference states.
In ( 6) and ( 7), we neglected the initial perturbation and obtained (9), which is related to CNOP-P.With the given reference states and the constraint C  of the model parameter perturbation, we can obtain the CNOP-P by solving (9).The Ref- is a steady state for (10), and the objective function   is always equal to zero, regardless of the model parameters.That is, parameter perturbation cannot cause a departure from the steady reference state Ref-.Consequently, there is no CNOP-P for Ref-.The reference states Ref- 1 and - 2 are time dependent and have one CNOP-P; the CNOP-P of Ref- 1 is the same as that of Ref- 2 .For simplicity, we list only the CNOP-P related to  1 (Table 5).
It is clear from Table 5 that the CNOP-P also locates the boundary of the constraint.Furthermore, the perturbation components are not the same as the parameter perturbation components in the CNOP patterns that were obtained above.This finding further demonstrates that the CNOP for a given reference state is not a simple combination of the corresponding CNOP-I and CNOP-P.As a comparison, we list in Table 6 the CNOP of the Ref- 1 for the constraint  = 0.4 and  = 0.3 and the simple combination of the corresponding CNOP-I and CNOP-P.It is shown that the CNOP-I in the simple combination is different from the initial perturbation component in the CNOP pattern.Additionally, the corresponding CNOP-P is not the parameter perturbation component in the "CNOP." Furthermore, the CNOP causes a larger prediction error than does the simple combination of CNOP-I and CNOP-P (i.e., the value of the objective function ( 0 ;   )), which emphasizes the importance of CNOP calculation in predictability studies.

A Property of CNOP, CNOP-I, and CNOP-P
It has been demonstrated numerically that CNOP, CNOP-I, and CNOP-P for the coupled Lorenz model always locate the boundary of their respective constraints.Furthermore, Liu [18] proved theoretically that CNOP-I locates the boundary of the constraint for the model that depends continuously on initial perturbations.Next, we prove theoretically that the CNOP and CNOP-P also locate the boundaries of their respective constraints.

Theorem 1. The CNOP and its two special cases, CNOP-I and CNOP-P, locate the boundary of their respective constraints for the models whose perturbation solutions depend continuously on initial perturbations and parameter perturbations.
Here, we prove this property of CNOP.Without loss of generality, the above initial perturbations of ( 15), ( 0 ,   ), are assumed to be the initial perturbations superimposed on the origin, where the origin is regarded as the equilibrium basic state, that is, when  0 = 0,   = 0,    ( 0 ;   ) = (;  0 ,   ) = 0.With this assumption, the optimization problem related to CNOP becomes Using ( 16), the CNOP-I ( 0 ,    ) of ( 15) can be obtained; that is, ( 16) defines the CNOP-I of (15).In fact, this CNOP-I is the CNOP of (1).Liu [18] proved that CNOP-I locates the boundary of the constraint if the relevant model depends continuously on the initial perturbation.Furthermore, it has been assumed that the perturbation equation () =    ( 0 ;   ) depends continuously on initial values ( 0 ;   ).Therefore, according to Liu [18], the CNOP of (1), that is, the CNOP-I of (15), locates the boundary of the constraint.

𝐽 (𝑝
To facilitate this discussion, if we signify the constraint  = {( 0 , ) |  0 = 0,  ∈   }, then  is a closed set including the origin, indicating that the initial perturbations of ( 15) are constrained by the closed set . Furthermore, it has been assumed that (15) depends continuously on the initial perturbation.According to Liu [18], the CNOP-I of ( 15), (0,    ), is on the boundary of the constraint .Equivalently, the CNOP-P    locates the boundary of   .Therefore, if we further assume that (1) depends continuously on parameter perturbations, CNOP-P usually locates the boundary of its constraint.

Summary
The CNOP is a useful tool for studying the limits of predictability in numerical weather forecasting and climate prediction.The CNOP has two special cases: one is the CNOP-I, which only relies on initial perturbations and has the largest nonlinear evolution at the time of prediction, while the other, CNOP-P, is merely related to the parameter perturbations, which causes the largest departure from a given reference state at a given future time.To find the CNOPs of a model, we reduce the model into an optimization problem.In this paper, we explore the CNOP, CNOP-I, and CNOP-P of a coupled Lorenz model and study their dependence on the reference state.We find that the CNOP, CNOP-I, and CNOP-P always locate the boundary of their respective constraints; we have proved this theorem logistically.Furthermore, we demonstrate that the CNOPs are dependent on the properties of the given reference states.For example, the reference state Ref- is steady and, together with the -axis, acts as a symmetric axis for the coupled Lorenz model system, which means that the Ref- has two symmetric CNOPs about the -axis.The reference states Ref- 1 and Ref- 2 are unsteady but symmetric about the -axis.This property of these two reference states causes the two reference states to have one CNOP; furthermore, these two CNOPs are symmetric with respect to the -axis, similar to their reference states.In addition, we demonstrate that the CNOP is not a simple combination of CNOP-I and CNOP-P, which suggests further investigation of the optimal combined mode of the initial perturbations and model perturbations, that is, the CNOP for predictability studies of numerical weather forecasting and climate predictions.
In this paper, we use a simple model to demonstrate CNOPs.In numerical weather forecasting and climate prediction, the models are generally complex.Therefore, to meet the requirements of realistic applications, we should solve the CNOPs of more complex models.The resulting CNOPs should also prove useful in improving the accuracy of weather and climate forecasts.If mathematicians and meteorologists collaborate, their skills in forecasting weather and climate may be greatly improved.
1 and Ref- 2 are unsteady and may not be a symmetric axis, in which case symmetric CNOPs do not exist.Nevertheless, we notice that the Ref- 1 and Ref- 2 are symmetric about the axis.Are the CNOPs of these two reference states symmetric about -axis?To address this question, we compute the CNOPs for Ref- 1 and Ref- 2 .

Table 3 :
CNOP-Is for initial condition  with different initial constraints.

Table 4 :
CNOP-Is for the reference state with initial condition  1 .

Table 6 :
The CNOP of the Ref-C1 for  = 0.4 and  = 0.3 and the simple combination of the corresponding CNOP-I and CNOP-P.