Toolgraph Design of Optimal and Feasible Control Strategies for Time-Varying Dynamical Systems

The paper presents a new method for designing optimal and feasible control strategies for time-variant dynamical processes. The key point of the presented idea lies in utilizing a flow graph structure for representing pertinent properties of the autonomous dynamics of a given dynamical process in a time-and-state space, which is composed of certain elementary segments. The structure is referred to as a time-and-state space toolgraph. In the procedure, each segment of the temporary state space is assigned a node of the time-and-state space toolgraph. The flow values are proportional to the cost of driving the operational point of the dynamical process between the centers of adjacent segments. Any of the discrete optimization algorithms can be applied to search for a cheapest path connecting the initial and terminal points of the sought optimal piecewiselinear trajectory of the operational points in the considered time-and-state space. Additional assumptions or restrictions concerning arbitrary forbidden zones for the operational points can be easily taken into account. In such cases the nodes representing the segments partially or entirely belonging to the finite forbidden zones are deposed from the toolgraph structure.


Introduction
The general idea of representing a continuous workspace by means of a graph structure has recently been explored, for instance, for searching optimal trajectories of robot movements in 2D 1 and 3D 2 workspaces, as well as for designing optimal control in multidimensional spaces 3 .
The presented method is a consequent extension of our research track of designing optimal and feasible control by means of discrete optimization methods 4-6 .In particular, certain deterministic time variations of the controlled process are taken into consideration by extending the common state space with an additional time dimension.
The algorithm consists in finding a trajectory in such an extended state space called henceforth the time-state space , which should satisfy two principal prerequisites.First, the sought trajectory circumvents certain finite areas called the forbidden zones which can, for instance, represent some failures previously identified.Second, the autonomous dynamics of the process is used in the process of minimizing the adopted indicator of control cost, which is a part of the problem definition and an input to a corresponding minimization procedure.The procedure starts with segmenting the defined operational workspace, which is a subset of the time-and-state space taken into consideration during the process of optimal trajectory design.For each segment a set of representative values is determined.The representative values representatives are all the quantities which are necessary to calculate the cost of an operational point movement in a given segment area.The next step is based on a formulation of a flow graph structure called toolgraph , which reflects the arrangement of the segments along with their appropriate representatives.Any discrete optimization algorithm that performs optimal search for the cheapest path can be utilized for searching the time-and-state space toolgraph for the cheapest path connecting the initial and terminal nodes representing the segments, which contain initial and terminal points of the sought trajectory, respectively.This paper is organized as follows: Section 2 gives a formal description of the presented idea and in Section 3 an extended example is presented.The paper ends by the Conclusions section.

Problem and Solution Description
This section provides some basic definitions confer 5 and gives a raw description of the presented method.

Problem Statement
Definition 2.1 autonomous and forced dynamics .A dynamical and time-variant process is described by the following function: , where D i , for i 1, . . ., l, represents a range along the axis x i , which is taken into account while seeking an optimal trajectory, is said to be an operational subspace.
Definition 2.3 forbidden zone Z .A subset Z of P prohibited for operational points is referred to as a forbidden zone.This means that the sought optimal trajectory cannot enter it.
Definition 2.4 system trajectory E u .Any sequence E u of consecutive states of a given dynamical system for a feasible control input u from a set U is said to be a system's trajectory or a trajectory of operational points in the system state space P \ Z.
Definition 2.5 transition vector Λ .A transition vector Λ is an ordered set of two elements where x 0 is the first element and x k is the last element of the sought optimal trajectory.Definition 2.6 cost function J E u of the system and control trajectory .Let A be a set of all trajectories E ∈ A ⊂ P \ Z.Any real function of the class A → R is said to be a cost function J E u of this trajectory.
Definition 2.7 optimal trajectory E * .Let Ξ ⊂ A be the subset of all possible trajectories that perform the desired transition Λ i.e., which start at x 0 and terminate at x k .We say that a trajectory E * is costly optimal if it satisfies the following conditions: Definition 2.8 autonomous dynamics map M .The image ẋx of the whole space P in the transformation f x is said to be an autonomous dynamics map M ẋx .
Our ultimate objective is to find the optimal and feasible control u * t along with its corresponding state trajectory E * E u * t ∈ P \ Z , which implements the complete transition Λ of the dynamical system 2.1 from its initial state x t 0 x 0 to the target state x t k x k and at the same time minimizes the cost functional of the following form: where β i , i 1, . . ., m are nonnegative factors often used for weighting, and t k is a suitable transition-time interval resulting from the applied optimal control procedure.

Solution Method
For the dynamical systems described in Definition 2.1, we consider its simple discrete-time variability of the computationally-convenient jump type allowing for the state-space system approximation scheme depicted in Figure 1.
Leaving apart the initial conditions defined for a certain t 0 , let us consider consecutive continuous-time computational time moments t n , for n 1, . . ., N, with t n − t n−1 h Δt, being a computation step or a simulation insight , where N ∈ N is a discrete-time horizon under consideration.Definition 2.9 temporary workspace P n .At each of the above-mentioned computational moments, a separate temporary duplicate P n of the operational subspace P will be associated, as shown in Figure 1.
Definition 2.10 snapshots M n of the dynamics map .The multiplication of the subspace P allows us to define separate snapshots M n of the dynamics map M assigned to consecutive time moments n 0, . . ., N.
The map snapshots M n are thus assigned to each of the temporary workspaces P n .This is, however, difficult to be portrayed on the composite scheme given in Figure 1.Instead, there are only shaded planes illustrating this effect.A better portray of map snapshots is given in the following example see Figure 2 .
Definition 2.11 continuous-time dynamic system with discrete-time variability .The temporary potentially time-variant system dynamics composed of the autonomous and forced dynamics can be defined in the computational moments as ẋ t n ẋx t n ẋu t n 2.9 which by the power of 2.3 and 2.4 allows for simple modeling of the system variability which can be arbitrary induced by redefining the functions f x and f u .
Note that the snapshots M n of the map are included in 2.9 as ẋx , whereas the control part ẋu represents the forced component 2.4 of the complete state derivative ẋ.Clearly, this means that not just the snapshot M n forms the derivative of the state.Definition 2.12 process computation .The procedure of calculating the operational-point trajectory in the state-space for the assumed discrete-time variability model which can also be treated as a kind of real-system approximation consists of the following derivative-based prediction and numerical integration steps: i take ẋ t n and treat it as a local estimation of the system dynamics for the future time interval t n , t n 1 ,

Vector of the local dynamics
Trajectory segment approximation ii calculate the next state vector for the time moment t n 1 as

2.10
Note that the rectangular integration scheme applied above for modeling the timevariability of dynamical systems is widely utilized in the procedures of computer simulation of variant or invariant continuous-time systems 7 .
Definition 2.13 segmentation of the temporary operational subspaces .Segmentation of the temporary operational subspace P n consists in dividing it, for the time moments t n , n 0, . . ., N, into a set S n of S segments denoted as Φ n j , for j 1, . . ., S, according to the following relation: where each pair of segments for any discrete time n satisfies the following separability condition: 12 for i, j 1, . . ., S; i / j, where δ denotes the closure of the corresponding set.Clearly, the size of the segments is typically selected in accordance with some fidelity criteria of the applied discrete approximation of the original process.
For the computational purposes we need a definition of a segment neighborhood and a generic concept of a distance norm .Let us thus assume the common Euclidean norm • e and the following definition.
if their geometrical centers x B and x A satisfy a simple neighboring condition: where • e means the Euclidean norm, and r is a fixed neighborhood size.
Definition 2.15 representatives .Any variable used in optimization and assigned to a segment is said to be a representative.
The autonomous dynamics map of Definition 2.8, being the derivative of the vector function x t , makes a principal source of candidates for a rational selection of the segment representatives.Note that in accordance to the common idea of domain and codomain of the transformation f x , from a practical implementation viewpoint, the map means a kind of doubling of the problem dimensioning with respect to P n .This fact is merely represented in Figure 1 by the colors or rather shades applied to different temporary subspaces P n .When the local evolution ẋ of some of the coordinates of the state vector is not relevant for the optimization considered, the co-domain can be reduced.At the same time the designer can introduce other representatives relevant for the optimization procedure and criterion.Definition 2.16 predecessor and successor workspaces .For any pair P n , P n 1 , P n is called a predecessor and P n 1 is referred to as a successor on the time scale .Definition 2.17 time-and-state operational space .The time-and-state operational sub space, concerning all the time moments t n , n 0, . . ., N, is composed as follows: P {P 1 , P 2 , . . ., P n , . . ., P N }.

2.14
In spite of the above definition of the global time-and-state operational space, and according to the scheme depicted in Figure 1, the principal optimization takes place in particular temporary workspaces P n .Clearly, based on the procedure judgment shown in Figure 1, the time domain has to be treated separately.
Definition 2.18 state-space graph G n .A flow graph representing the arrangement of the segments of the temporary operational workspace P n , n 0, . . ., N, is said to be a state-space graph.According to the common idea 8 of flow graphs: we interpret its basic structure as a set W n of nodes, each of which is assigned a set of the values of the representatives of its corresponding segments.Only the edges connecting two nodes representing segments located in their mutual neighborhood form a set D n of the edges.Each edge d ∈ D n is assigned a flow value k ∈ K n .
Definition 2.19 time-and-state space toolgraph .According to the definition of the time-andstate operational space, and based on the state-space graphs of all the workspaces P n , a timeand-state space toolgraph is composed by adding costless, directed edges connecting all the pairs of equivalent in the subspace geometry segments in the predecessor and successor workspaces.

Algorithm Description
Basically, the presented algorithm takes the above-defined principal notions the time-andstate space P composed of the temporary operational workspaces P n , the transition vector Λ, and the segmentation form to minimize the adopted cost function J E u .Optionally, we can determine the feasible control set U and the forbidden zone Z.The latter can be used to represent some constraints portraying, for instance, faults previously detected in the system.
Definition 2.20 the toolgraph optimization algorithm .Initialization with the input data: dynamic process description, operational time-and-state space, forbidden zones, initial and terminal trajectory points, and time horizon is as follows.
i As can be seen in Definition 2.20, we start the procedure by segmenting the temporary subspaces P n which results in a limited set of S segments, for n 0, . . ., N. Each segment of a set of representatives is assigned.Certainly, the size of the segments is chosen in accordance to an assumed discretization error, or fidelity index discussed later on in the examples .
Next, the state space graphs are formed.According to Definition 2.18, each node from W n is associated with a certain segment of P n .Its edges connect only nodes representing the segments neighboring in P n .The edge flow values are determined according to the adopted function cost.Consequently, the time-and-state space toolgraph is built as a composition of the "consecutive" state-space graphs according to Definition 2.19.
For practical optimization problems, any discrete optimization algorithm, which is suitable for finding the cheapest path between two selected nodes, can be utilized.If such a path exists, it represents a sequence of segments, which should be visited by the optimal trajectory.This sequence of segments implies a reference trajectory, which can be tracked by an executive controller in a procedure of a suboptimal fulfillment of the stated control task.

Optimization Examples
This section provides an illustrative example which utilizes the above-mentioned formal definitions and the optimization algorithm.The minimized criterion is given by 2.8 with the restriction that the resulting trajectory is piecewise linear in the considered time-and-state space.The example concerns a two-dimensional time-variant dynamical process described in the following subsection.

Process Description
The exemplary process is described by the following set of equations: where The autonomous dynamics map of the described process is shown in Figure 2, where certain three B, C, D centers of attraction can be seen, which circulating with the angular velocities Ω B , Ω C , Ω D , resp.around the center A are the source of process time-variability.Additionally, the forbidden zones Z A , Z B , Z C , Z D are defined in the forms of circular areas, which have the radii r A 10, r B 20, r C 15, and r D 10, respectively.The midpoints of the forbidden zones cover the midpoints of the respective centers of "attraction."

Algorithm Implementation
In theory, an infinite time horizon of optimization is a simple and trivial solution for the time horizon setting.In computer applications, we prefer the concept of "sufficiently" large horizons, which is determined according to our prior knowledge on the optimization process and the limitations of available computing power, whereas the time step size was chosen empirically so as to obtain the desired "granularity" of the solution.
According to Definition 2.13, the segmentation procedure consists of dividing the operational subspaces P n , n 0, . . ., N, being, in this case, hypercubes described by D 1 × D 2 −250, 250 × −250, 250 , into the set of segments of diameters r x 1 10, r x 2 10 , all located along the time axis at a step size Δt 10.For each segment the representative values are calculated as the averages of the autonomous dynamics computed for particular directions: where F DYN n i,k denotes the kth sample of the coordinate x i of the autonomous dynamics map for the discrete time moments n, and N p 25 denotes the number of samples per segment in this following examples.
The fidelity of the applied discrete representation for the segment Φ n j,i is assessed as If e Φ n j,i exceeds some given threshold, which can be an empirically estimated value, we assume that the segment Φ n j,i along with its representatives constitutes an infeasible representation of the autonomous dynamics in the area of the time-and-state space occupied by this segment.In such cases either the size of the segment is changed or the segment is included to the forbidden zone as an area of high dynamics .
Having defined the neighborhood of the segments, the construction of the state-space graphs and time-and-state space toolgraph comes according to Definitions 2.18 and 2.19, respectively.The flow values are determined as the cost of driving the operational point between each two neighboring segments according to 2.8 .
The last stage consists in searching the above-defined time-and-state space toolgraph for the cheapest path connecting the initial and the terminal node.We can use, for instance, the Dijkstra's algorithm 9 for this purpose.

Problem 1
The first problem concerns finding a trajectory of the operational point, which minimizes 2.8 , with the initial location at x 1o −250, x 2o −250, t 0 0 and the terminal point x 1k , x 2k , t k satisfying the following condition: where the assumed time horizon is t N 2000.The results are presented in Figure 3.The total cost indicator is equal to J 1 43.454 × 10 −2 .It can be clearly seen that the optimal strategy consists in keeping the operational point at the initial location till a certain time moment t * 1405 and then to start the transition process.

Problem 2
In this case the time horizon was shortened to t N 1600, which resulted in a higher value J 2 52.494 × 10 −2 of the total cost indicator as compared to the one of Problem 1.The remaining parameters are the same as in Problem 1.The results are depicted in Figure 4.

Problem 3
Further shortening of the time horizon t N 1000 leads to a greater control cost J 3 150.744× 10 −2 with the trajectory of the operational point depicted in Figure 5.

Problem 4
This time the problem of driving the operational point from the initial condition x 1o 0, x 2o 0, t 0 0 to the terminal point x 1k > 180, x 2k > 180, t k ≤ t N is considered.The time horizon is t N 1000.The resulting total cost is equal to J 4 275.368× 10 −2 .The designed trajectory is presented in Figure 6.

Problem 5
This case is a variation of Problem 4 with the time horizon lengthened to t N 4000, which resulted in a reduction of the total cost J 5 202.905 × 10 −2 .The computed trajectory is presented in Figure 7.

Problem 6
The last example presents the design of the optimal trajectory with the initial point at x 1o 2.5, x 2o −17.5, t 0 0 and the terminal point satisfying 3.7 with the time horizon t N 4000.The total cost J 5 24.752 × 10 −2 .The gained results are depicted in Figure 8.

Conclusions
This work presents an effective concept of designing optimal and feasible control strategies for time-varying dynamical processes based on discrete optimization approach and its solutions.
After segmenting the composed time-and-state workspace of the process dynamics, a time-and-state space toolgraph made of temporary state-space graphs is constructed that represents all the pertinent properties of the autonomous dynamics of the considered dynamical process.Additional assumptions or restrictions concerning arbitrary forbidden zones for the operational points can be easily taken into account.Numerical computations are performed by means of any discrete optimization algorithm which searches for the cheapest path connecting the node initial and terminal nodes.The presented work has been implemented based on the basic form of the Dijkstra's algorithm, whose performance can be improved by applying various modifications given in the literature, for instance, in 10 .
The resulting cheapest path describes an optimal piecewise-linear trajectory of the operational points in the considered time-and-state space.Note that such a solution may need some kind of smoothing for most real-time process cases.
A clear suboptimal property of the proposed approach results from the fact that the computed reference trajectory can only pass through the centers of the segments.In this context the size of segments has a key influence on the quality of the ultimate optimal solution being sought.
Thus, in general, this interdisciplinary paper concerns research into formal optimization methods and tools.Namely, it shows how to constructively predefine the control optimisation problem or to convert complex optimal control problems for nonlinear timevariant objects with constraints into a graph structure ready to be processed by any of the discrete optimization methods, in particular, by the standard numerical methods of graph optimisation.In the light of theory application to problems arising in engineering, this method can be used for on-or off-line numerically solved practical optimal control or pathfinding problems for time-varying objects or environments.
Though this paper is an investigative proposition of a new numerical optimization method, an abstract but nontrivial and imaginative example of several variations has also been given to illustrate the effectiveness of the proposed methodology.The example presented inclusive of a system of variant attraction centers is a rough analog to the solar system.Other business and industry applications are to be developed in order to give a wider proof of the advantages of the proposed method in a confrontation with the scientific challenges of the real world.We also hope that this paper should encourage scientists and engineers to apply both this approach and the discrete optimisation methods.
Complexity of the presented method principally results from the complexity of the algorithm applied for searching the toolgraph.Some clues concerning the possibilities of improving the graph search process can be found in 11, 12 , for instance.It is clear that the difficulty of the proposed method is shaped mostly by the numerical complexity of searching the T-graph.In a straightforward tactic one makes use of the Dijkstra algorithm O n 2 .Other, mostly heuristic enhancements like the renowned algorithm A * can also be employed in order to improve the efficiency of the proposed numerical optimisation procedure.
In spite of the presented feasibility of the approach, a general problem of the so-called curse of dimensionality has always to be taken into account in all research of this type.This means that there are limits which render such approaches infeasible in some problems of practical interest.Nevertheless, nowadays, in view of the persistent progress in computer technology, finding constructive methods of solving problems appears to be most substantial in science, technology, and engineering.

1 Figure 3 :
Figure 3: Resulting optimal trajectory for Problem 1 solid black line ; the forbidden zones of the respective centers of attractions: A, B, C, D colored areas .

1 Figure 4 : 1 Figure 5 : 1 Figure 6 :Figure 7 :
Figure 4: Optimal trajectory for Problem 2 solid black line ; the forbidden zones of the respective centers of attractions: A, B, C, D colored areas .

1 Figure 8 :
Figure 8: Optimal trajectory for Problem 6 solid black line ; the forbidden zones of the respective centers of attractions: A, B, C, D colored areas .