A Probabilistic Approach to Control of Complex Systems and Its Application to Real-Time Pricing

Control of complex systems is one of the fundamental problems in control theory. In this paper, a control method for complex systems modeled by a probabilistic Boolean network (PBN) is studied. A PBN is widely used as a model of complex systems such as gene regulatory networks. For a PBN, the structural control problem is newly formulated. In this problem, a discrete probability distribution appeared in a PBN is controlled by the continuous-valued input. For this problem, an approximate solution method using a matrix-based representation for a PBN is proposed. Then, the problem is approximated by a linear programming problem. Furthermore, the proposedmethod is applied to design of real-time pricing systems of electricity. Electricity conservation is achieved by appropriately determining the electricity price over time. The effectiveness of the proposed method is presented by a numerical example on real-time pricing systems.


Introduction
Analysis and control of complex systems such as power systems and gene regulatory networks are one of the fundamental problems in control theory of large-scale systems.In order to deal with such complex systems, it is one of the appropriate methods to approximate a complex system by a discrete abstract model (see, e.g., [1]).On the other hand, human decision making is also complex and is modeled by a discrete model (see, e.g., [2]).Thus, in analysis and control of complex systems and those with human decision making, a discrete model plays an important role.
Several discrete models such as Petri nets, Bayesian networks, automata-based models, and Boolean networks have been proposed so far (see, e.g., [3]).In this paper, we focus on a Boolean network (BN) [4].In a BN, the state is given by a binary value (0 or 1), and the dynamics are expressed by a set of Boolean functions.Since Boolean functions are used, it is easy to understand the interaction between states.In the field of theoretical biology, there is a criticism that a BN is too simple as a model of gene regulatory networks (see, e.g., [5]), but a BN can be relatively applied to largescale systems.In addition, since the behavior of complex systems is frequently stochastic by the effects of noise, it is appropriate that a Boolean function is randomly decided at each time among the candidates of Boolean functions.Thus, a probabilistic BN (PBN) has been proposed in [6].Furthermore, a context-sensitive PBN (CS-PBN) in which the deciding time is randomly selected has been proposed as a general form of PBNs [7,8].In this paper, we adopt a probabilistic Boolean network (PBN) as a mathematical model of complex systems.
For a given PBN, we consider the structural control problem (see, e.g., [9][10][11]).In this problem, a discrete probability distribution is controlled.For example, in [9], a discrete probability distribution at each time is selected among a given set.In this paper, we consider fine control of a discrete probability distribution by using the continuous-valued input.For a newly formulated problem, we propose an approximate solution method.First, a matrix-based representation of BNs proposed in [12] is extended to that of PBNs.Next, using the obtained representation, the original problem is approximated by a linear programming (LP) problem.
Furthermore, as one of the applications, we consider a design method of real-time pricing systems (see, e.g., [13][14][15][16]).A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve.In general, a real-time pricing system consists of one controller deciding the price at each time and multiple electric customers such as commercial facilities and homes.If electricity conservation is needed, then the price is set to a high value.Since the economic load becomes high, customers conserve electricity.Thus, electricity conservation is achieved.In the existing methods, the price at each time is given by a simple function with respect to power consumptions and voltage deviations and so on (see, e.g., [16]).To the best of our knowledge, decision making of customers has not been explicitly considered so far.In order to realize more precisely pricing, it is necessary to use a mathematical model of customers.Thus, decision making of customers is modeled by a PBN, and the problem of finding the price at each time is formulated as a structural control problem.The price corresponds to the continuousvalued input.By a numerical example, the effectiveness of the proposed method is presented.
The proposed framework provides us a basic method for control of complex systems using PBNs.
Next, we explain a probabilistic Boolean network (PBN) (see [6] for further details).In a PBN, the candidates of  ()  are given, and for each   , selecting one Boolean function is probabilistically independent at each time.Let denote the candidates of  () .The probability that  ()  is selected is defined by Then, the following relation must be satisfied.Probabilistic distributions are derived from experimental results.Finally, N  ,  = 1, 2, . . .,  are defined by We present a simple example.
Example 1.Consider the PBN in which Boolean functions and probabilities are given by ) 1 = 0.8,  (1)  2 = ¬ 3 () ,  (1)  2 = 0.2, where (1) = 2, (2) = 1, and (3) = 2 hold, N 1 = {3}, N 2 = {1, 3}, and N 3 = {1, 2} hold, and we see that the relation ( 5) is satisfied.Next, consider the state trajectory.Then, for (0) = [0 0 0] ⊤ , we obtain Mathematical Problems in Engineering In this example, the cardinality of the finite state set {0, 1} 3 is given by 2 3 = 8, and we obtain the state transition diagram of Figure 1 by computing the transition from each state.In Figure 1, the number assigned to each node denotes  1 ,  2 ,  3 (elements of the state), and the number assigned to each arc denotes the transition probability from some state to other state.Note here that, for simplicity, the state transition from only

Problem Formulation
In this section, we formulate the control problem studied in this paper.In the conventional control problem, the control input is added to a given Boolean function.For example, the control input is added as follows:  () ([  ()] ∈N () , ()), () ∈ {0, 1} 1 .In general, we assume that the value of the control input can be arbitrarily given.However, there is a possibility that there exists no control input satisfying this assumption.In control of gene regulatory networks, a structural control (or structural intervention) method for PBNs has been proposed so far (see, e.g., [9,10]).For example, in [9], the discrete probabilistic distribution is switched at each time.In other words, the discrete probabilistic distribution is selected from the set of candidates.On the other hand, in complex systems such as gene regulatory networks, power systems, and social systems, it will be desirable to consider a weaker control method.Thus, in this paper, we consider fine control of probabilities in a discrete probabilistic distribution.This control method can be regarded as a kind of structural control methods.
In the structural control problem formulated here, we assume that the probability  ()   in ( 4) is given by where The set [  ,   ] expresses the input constraint, and   ,   ∈ R 1 are given in advance.The parameter  ()   expresses elasticity of the probability to the control input.Finding  ()  and  ()  is the important problem, and will be focused on in future efforts.Of course, we must find   () such that  ()   () satisfies ( 5) and In addition, the dimension of the control input may be less than the dimension  of the state.Under the above preparation, we consider the following problem.
Problem 2. Suppose that for the PBN with (9), the lower and upper bounds of input constraints   ,   , and the initial state (0) =  0 are given.Then, find a control input sequence under the constraints ( 5) and (10), where ,   ∈ R 1× ,  ∈ R 1× are weighting vectors whose element is a nonnegative real number, and [⋅ | ⋅] denotes a conditional expected value.
The linear cost function (11) is appropriate from the following reason.For a binary variable  ∈ {0, 1}, the relation  2 =  holds.That is, in the cost function, the quadratic term such as  2  () is not necessary.According to the result in [17], Problem 2 can be rewritten as a polynomial optimization problem.However, in the case of large-scale PBNs, it will be difficult to solve a polynomial optimization problem.In this paper, an approximate solution method for Problem 2 is proposed.
Hereafter, the condition (0) =  0 in the conditional expected value is omitted.

Solution Method
In this section, we derive an approximate solution method for Problem 2. First, a matrix-based representation for PBNs is derived.The obtained representation is an extension of a matrix-based representation for BNs proposed in [12].Next, using the matrix-based representation, an approximate solution method for Problem 2 is derived.First, we explain the outline of a matrix-based representation by using a simple example.Example 3. Consider the following BN: where = {1}, and N (3) = {1, 2}.Then, we can obtain the truth table for each   ( + 1).See Tables 1 and 2.
From these truth tables, we can obtain the following matrixbased representation: where each element of  () ,  = 1, 2, 3 is given by a binary value (0 or 1), and a sum of all elements in each column of  () is equal to 1.
Such a matrix-based representation has been proposed in also [18,19].However, in the representation proposed in [18,19], matrices with the size of 2  × 2  must be manipulated ( is the dimension of the state).In the matrixbased representation proposed in [12], matrices with the size of 2 × 2 |N  | are manipulated for each   .Thus, the proposed representation enables us to model a BN using matrices with the smaller size.
Consider a general case.Define ) .
Then, the matrix-based representation for   ( + 1) is given by where . The matrix  () can be derived from the following procedure.
Next, consider extending the matrix-based representation of BNs to that of PBNs.First, using a simple example, we explain the outline.
In a similar way to Problem 2, Problem 5 is rewritten as a polynomial optimization problem.In this paper, we focus on the structure of ⨂ ∈N  [  ()] and derive the relaxed problem for Problem 5.The relaxed problem is reduced to a linear programming (LP) problem, which can be solved faster than a polynomial optimization problem.
First, we present an example.
Example 6.Consider the matrix-based representation obtained in Example 4. We remark that the discrete probabilistic distribution for each   is independent.Then, in (17), we can obtain In a similar way, we can obtain In addition, holds.The obtained equalities are linear with respect to and and and Hence, these can be used as constraints in the relaxed problem.
Next, consider a general case.Define Then, (18) can be rewritten as where For . . .(26) From   () :=   ()  (), we can obtain In addition, we introduce the following constraints: Thus, we can obtain the following problem as a relaxed problem of Problem 5.
Problem 7. Find (0), (1), . . ., ( − 1) minimizing the cost function (11) subject to ( 5), (10), and ( 23)-( By a simple calculation, Problem 7 can be equivalently rewritten as the LP problem with   (),   (), and   () as decision variables.By solving Problem 7, we can evaluate the lower bound of the optimal value of the cost function in Problem 7. In this paper, only an approximate solution method is provided.However, since the control input can be obtained by solving an LP problem, Problem 7 can be solved fast.
Finally, according to the receding horizon policy (see, e.g., [20]), we present the procedure of model predictive control (MPC).
Step 4. Apply only () to the system.

Application to Design of Real-Time Pricing Systems
In this section, we consider a design method of real-time pricing systems as an application of structural control of PBNs.First, the outline of real-time pricing systems of electricity is explained.Next, the PBN-based model of realtime pricing systems is derived.Finally, a numerical example is presented.5.1.Outline. Figure 2 shows an illustration of real-time pricing systems studied in this paper.This system consists of one controller and multiple electric customers such as commercial facilities and homes.For an electric customer, we suppose that each customer can monitor the status of electricity conservation of other customers.In other words, the status of some customer affects that of other customers.For example, in commercial facilities, we suppose that the status of rival commercial facilities can be checked by lighting, Blog, Twitter, and so on.Depending on power consumption, that is, the status of electricity conservation, the controller determines the price.If electricity conservation is needed, then the price is set to a high value.Since the economic load becomes high, customers conserve electricity.Thus, electricity conservation is achieved.The price does not depend on each customer and is uniquely determined.

Model. Consider modeling the set of customers as a PBN.
The number of customers is given by .We assume that the state of customer  ∈ {1, 2, . . ., } is binary and is denoted by   .The state implies where () ∈ [, ] ⊆ R 1 is the control input corresponding to the price.The Boolean functions  ()  1 and  () 2 imply that customer  forcibly conserves (or does not conserve) electricity.In these cases, time evolution of the state does not depend on the past state.The Boolean function  ()  3 implies that the state is not changed.The Boolean function  ()  4 implies that the state of customer  is changed depending on the other customers.The Boolean function  ()  5 implies that the state of customer  is changed depending on the leader.Thus, decision making of customers can be modeled by a PBN.The above Boolean functions are an example of models for decision making.Depending on real situations, we may use other Boolean functions.
For the PBN-based model obtained, we consider the following problem: find a time sequence of the price such that customers conserve electricity as much as possible.However, it is not desirable that the price is too high.
The condition that customers conserve electricity as much as possible can be characterized by [  ].In other words, power consumption is expressed by [  ].Hence, this problem can be formulated as Problem 2 by appropriately setting the weights  and .
Next, we present the computation result.Here, Problem 7 was solved once.Figure 3 shows trajectories of [  ()].Figure 4 shows trajectories of the control input (the price).From these figures, we see that [  ] becomes small by fine adjustment of the control input.In this example, the expected value of each state converges to 0.32.
In addition, when the obtained control input is applied to the system, the value of the cost function in Problem 2 was 79.4311.In the case of () = 0.3 (i.e., the constant input), the value of the cost function in Problem 2 was 85.3581.In the case of () = 0.7, the value of the cost function in Problem 2 was 87.0247.From these values, we see that the obtained control input is more effective than trivial control inputs.Furthermore, in order to verify the optimality, consider the case of  = 2.The optimal control input was derived as (0) = 0.7 and (1) = 0.3 by solving the polynomial programming problem, which is equivalent to Problem 2. On the other hand, the control input obtained by solving Problem 7 was (0) = 0.7 and (1) = 0.52.Thus, there is a possibility that the control input obtained by solving Problem 7 is not optimal for Problem 2.
Finally, we discuss the computation time for solving Problem 7. The computation time was 0.6 sec for  = 15 and 0.03 sec for  = 2, where we used IBM ILOG CPLEX 11.0 as the LP solver.The computation time for solving the polynomial optimization problem for  = 2 was 232.2 sec, where we used SparsePOP [21] and MATLAB 32-bit version.In the case of  ≥ 3, owing to memory warning, the polynomial optimization problem cannot be solved.Thus, although Problem 7 is an approximation of the original problem, Problem 7 can be solved fast.

Conclusion
In this paper, we studied control of complex systems modeled by a probabilistic Boolean network (PBN).First, the structural control problem for a PBN was newly formulated.Next, an approximate solution method was proposed based on a matrix-based representation of Boolean functions.Finally, as an application, we considered design of real-time pricing systems of electricity.The proposed method provides us a new control method for complex systems.
There are several open problems.It is significant to consider a method for evaluating the accuracy of an approximation from the theoretical viewpoint.It is also significant to develop an identification method of Boolean functions and parameters  ()   ,  ()  in (9).Finally, it will be one of the interesting topics to apply the proposed method to several classes of PBNs, for example, a large-scale PBN with scale-free structure.

Mathematical
) denote the th element of   () and   (), respectively.Then, we can obtain2 |N  |

Figure 2 :
Figure 2: Illustration of real-time pricing systems.

Figure 3 :Figure 4 :
Figure 3: The expected value of the state.Some states are indistinguishable.