In previous work, attention was restricted to tracking the net using a backward method that knows the target point beforehand (Bellmans's equation), this work tracks the state-space in a forward direction,
and a natural form of termination is ensured by an equilibrium point
The shortest-path problem (see [
While it is possible to analyze such processes using
the existing classical theory through the Bellman's equation with the cost
criterion ([
Moreover, algorithms using Bellman's equation usually
solve the problem in two phases [
Tracking the state space in a forward direction allows the decision maker to avoid invalid states that occur in the space generated by a backward search. In most cases, the forward search gives the impression to be more useful than the backward search. The explanation is that in the backward direction, when the case of incomplete final states arises, invalid states appear causing problems.
Shortest-path problem [
We are concerned about the first case. However, we consider dynamical systems governed by difference equations described by Petri nets. The trajectory over the net is calculated using a discrete Lyapunov-like function. A Lyapunov-like function is considered as a distance function denoting the length from the source place to the equilibrium point. This work is concerned with the analysis of the decision process where a natural form of termination is ensured by an equilibrium point.
Lyapunov-like functions can be used as forward trajectory-tracking functions. Each applied optimal action produces a monotonic progress towards an equilibrium point. Because it is a solution to the difference equation, naturally it will lead the system from the source place to the equilibrium point.
It is important to note that there exist areas of research using Petri nets as modeling tool where the use of a Lyapunov-like function is inherent. For instance, the “Entropy” function is a specific Lyapunov-like function used in Information Theory as a measure of the information disorder. The “free Gibbs energy function” is a Lyapunov-like function used in molecular biology for calculating the energy change in a metabolic network.
This paper introduces a modeling paradigm for shortest-path decision process representation in Petri nets theory. The main point of this paper is its ability to represent the characteristics related only with the global system behavior, and those characteristics related with the trajectory-tracking behavior.
Within the global system behavior properties, we show notions of stability. In this sense, we call equilibrium point to the place in a Petri net that its marking is bounded and it is the last place in the net (sink).
In the trajectory-tracking behavior properties framework, we define the trajectory function as a Lyapunov-like function. By an appropriate selection of the Lyapunov-like function, it is possible to optimize the trajectory. By optimizing the trajectory, we understand that it is the minimum trajectory-tracking value (in a certain sense). In addition, we use the notions of stability in the sense of Lyapunov to characterize the stability properties of the Petri net. The core idea of our approach uses a nonnegative trajectory function that converges in decreasing form to a (set of) final decision states. It is important to point out that the value of the trajectory function associated with the Petri net implicitly determines a set of policies, not just a single policy (in case of having several decisions states that could be reached). We call “optimum point” the best choice selected from a number of possible final decision places that may be reached (to select the optimum point, the decision process chooses the strategy that optimizes the trajectory-tracking value).
As a result, we show that the global system behavior properties and the trajectory-tracking behavior properties of equilibrium, stability, and optimum-point conditions meet under certain restrictions: if the Petri net is finite, then we have that a final decision place is an equilibrium point.
The paper is structured in the following manner. The
next section discusses the motivation of the work. Section
In this paper, we consider dynamical systems in which
the time variable changes discretely, and the system is governed by ordinary
difference equations. Let us consider systems of first-order difference
equations given by
Lyapunov defined a scalar function
The main idea of Lyapunov is attained in the following
interpretation: given an isolated physical system, if the change of the energy
A system is stable [
At this point, it is important to note that the
Lyapunov-like function
Lyapunov-like functions [
From what we have stated before, we can deduce the
following geometric interpretation of distance [
A Lyapunov-like function can be considered as a distance function denoting the length from the initial state to the equilibrium point. It is important to note that the Lyapunov-like function is constructed to respect the constraints imposed by the difference equation of the system. In contrast, a Euclidean metric does not take into account these factors. For that reason, the Lyapunov-like function offers a better understanding of the concept of the distance required to converge to an equilibrium point in a discrete dynamical system.
By applying the computed actions, a kind of discrete vector field can be imagined over the search graph. Each applied optimal action yields a reduction in the optimal cost-to-target value, until the equilibrium point is reached. Then, the cost-to-target values can be considered as a discrete Lyapunov function.
In our case, an optimal discrete problem, the cost-to-target values are calculated using a discrete Lyapunov-like function. Every time a discrete vector field of possible actions is calculated over the decision process. Each applied optimal action (selected via some “criteria”) decreases the optimal value, ensuring that the optimal course of action is followed and establishing a preference relation. In this sense, the criteria change the asymptotic behavior of the Lyapunov-like function by an optimal trajectory-tracking value.
Usually, the criterion in optimization problems is related with the choice of whether to minimize or maximize the optimal action. If the problem is related with energy transformations, as is classically the case in control theory, then the criterion of minimization is applied. However, if the dilemma involves a reward, typical in game theory, then maximization is considered. In this work, we will arbitrary consider the criterion of minimization.
The Lyapunov-like function can be employed as a
trajectory-tracking function through the use of an operator, which represents
the criterion that selects the optimal action that forces the function to
decrease and approaches an infimum/minimum. It forces the function to make a
monotonic progress toward the equilibrium point. The Lyapunov-like function can
be defined, for example, as
To illustrate the shortest-path
problem, let us consider a grid world (see Figure The relative entropy or
Kullback-Leibler [
An illustrative example of finding the shortest path in a grid world.
Glycolysis pathway (see Figure Glycolysis can be informally explained from an energetic perspective as follows. The
initial amount of glucose may be represented as a ball at the top of an
irregular hill. Every time the ball bounces, the hill represents a reaction
state in the breakdown of the sugar process. Each bounce of the ball
corresponds to a change in free energy level. This energy change is modeled by
the Gibbs energy function which is a Lyapunov-like function. It is important to
note that bounces are irregular (reaching lower and higher energy levels) and
determined by the environment conditions. The final state (pyruvate) is
represented by the bottom of the hill where the ball reaches a steady state
(not bounces). Let us explain the Petri net dynamics of the system
model as follows. Continuing with the ball and hill explanation, let us suppose
that the ball, representing the product pyruvate, is at the bottom of the hill.
And let us suppose that there is no net force able to move the ball either up
or down the hill. That means that the reactions (forward and backward) are
evenly balanced. Therefore, the substances and products are in equilibrium, and
no net dynamics will take place. That is, “the metabolic network system
is in equilibrium.”
Glycolysis and pentose-phosphate pathways model.
We introduce the concept of decision process Petri
nets (DPPNs) by locally randomizing the possible choices, for each individual
place of the Petri net [ A decision process Petri net is
a 7-tuple
We adopt the standard rules about representing nets as
directed graphs, namely, places are represented as circles, transitions as
rectangles, the flow relation by arcs, and markings are shown by placing tokens
within circles [
The previous behavior of the
In Figure
The probability that
We set by convention for the
probability that
Finally, we have the trivial case when
there exists only one arc from
It is important to note that, by definition, the
trajectory-tracking function
Consider an arbitrary
Continuing with all the
Then, formally we define the trajectory-tracking
function The
trajectory-tracking function
OR-Path (see Figure
(Left): routing policy case 1. (Right): routing policy case 2.
AND-Path (see Figure
OR-Path Example.
AND-path example.
From the previous definition, we have the following
remark.
(i) Note that the Lyapunov-like function (ii) The iteration over for for
The continues
function
if
there exists an infinite sequence if
there exists a finite sequence for all
From the previous property, we have the following
remark.
In property 1 point 3, we state that
The
trajectory-tracking function
Proof comes straightforward from the previous definitions.
From Properties In Property Property
Intuitively, a Lyapunov-like function can be considered as trajectory-tracking
function and optimal cost function. In our case, an optimal discrete problem,
the cost-to-target values are calculated using a discrete Lyapunov-like
function. Every time a discrete vector field of possible transitions is
calculated over the decision process. Each applied optimal transition (selected
via some “criterion,” e.g.,
Biochemical pathway of the free energy profile of the
glycolysis and pentose-phosphate. The following was adapted from Biochemistry
Lehninger et al. [
A decision is taken and
The Conc-Path is calculated at
We are using
We will identify the global system properties of the
DPPN as those properties related with the PN.
The
decision process Petri net Let us
suppose that the DPPN is not finite. Then Let
Let
Let us consider systems of first ordinary difference
equations given by The The system ( The system ( A continuous function
Let us consider [
Then, we have the following results [ Let The
stability properties are preserved for the
following. (1) Practically stable. Let us suppose that (2) Stable. Suppose that system ( (3) Asymptotically stable. We know that system ( (4) Uniformly stable. Assume that the
comparison system is uniformly stable, meaning that
We will extend
the last theorem to the case of several Lyapunov functions. Let us consider a
vector Lyapunov function Let (1) Let us suppose that (2) From the continuity of If in the point 1 of the proof it is not true that
Then, we have the following result [ From
Theorem If If The
diamond is the stable form of carbon at extremely high pressures while the
graphite is the stable form at normal atmospheric pressures. Regardless of
that, diamonds appear stable at normal temperatures and pressures, but, in
fact, are very slowly converting to graphite. Heat increases the rate of this
transformation, but at normal temperatures the diamond is uniformly practically
stable.
For Petri nets, we have the following results of
stability [ Let Let us choose as our candidate
Lyapunov function Let The if-and-only-if relationship of
( The biochemical pathway of the
glycolysis (Figure An equilibrium point with
respect to a decision process Petri net The decision process Petri net It is important to underline that
the only places where the DPPN will be allowed to get blocked are those which
correspond to equilibrium points.
We will identify the trajectory-tracking properties of
the DPPN as those properties related with the trajectory-tracking value at each
place of the PN. In this sense, we will relate an optimum point the best
possible performance choice. Formally we will introduce the following
definition [ A final decision point An optimum point
Every decision process Petri net
In case that The monotonicity of
Then, we can conclude the following
theorem.
Let Let us
suppose that Let We have to show that Let We have that The decision process Petri net Let Let us suppose that Let If Then, it is not bounded. So, it is possible to
increment the marks of Then, it is not bounded and it is not a sink.
So, it is possible to fire some output transition to Let From
the previous theorem, we know that a final decision point is an equilibrium
point and since in particular
Let Let us
suppose that The complexity time Each path in Let We have to show that The
finite and nonblocking (unless (1) Let us suppose that the (2) Let us suppose that the
In this work, a formal framework for shortest-path decision process problem representation has been presented. Whereas in previous work, attention was restricted to tracking the net using a utility function Bellman's equation, this work uses a Lyapunov-like function. In this sense, we are changing the traditional cost function by a trajectory-tracking function which is also an optimal cost-to-target function for tracking the net. This makes a significant difference in the conceptualization of the problem domain. The Lyapunov method introduces a new equilibrium and stability concept in decision process.