The Scientific World Journal

Volume 2014 (2014), Article ID 968341, 8 pages

http://dx.doi.org/10.1155/2014/968341

Research Article

Verification and Optimal Control of Context-Sensitive Probabilistic Boolean Networks Using Model Checking and Polynomial Optimization

Koichi Kobayashi and Kunihiko Hiraishi

School of Information Science, Japan Advanced Institute of Science and Technology, Ishikawa 923-1292, Japan

Received 30 August 2013; Accepted 28 October 2013; Published 23 January 2014

Academic Editors: Y. Lai, R. Purohit, and T. Roegnvaldsson

Copyright © 2014 Koichi Kobayashi and Kunihiko Hiraishi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

One of the significant topics in systems biology is to develop control theory of gene regulatory networks (GRNs). In typical control of GRNs, expression of some genes is inhibited (activated) by manipulating external stimuli and expression of other genes. It is expected to apply control theory of GRNs to gene therapy technologies in the future. In this paper, a control method using a Boolean network (BN) is studied. A BN is widely used as a model of GRNs, and gene expression is expressed by a binary value (ON or OFF). In particular, a context-sensitive probabilistic Boolean network (CS-PBN), which is one of the extended models of BNs, is used. For CS-PBNs, the verification problem and the optimal control problem are considered. For the verification problem, a solution method using the probabilistic model checker PRISM is proposed. For the optimal control problem, a solution method using polynomial optimization is proposed. Finally, a numerical example on the WNT5A network, which is related to melanoma, is presented. The proposed methods provide us useful tools in control theory of GRNs.

1. Introduction

Control of gene regulatory networks (GRNs) is one of the significant topics in the field of systems biology, and is also one of the basics of therapeutic interventions (see, e.g., [1]) in the future. Furthermore, in recent years, the experimental result on control of GRNs has been obtained in [2]. That is, feedback control of synthetic biological circuits has been implemented, and the experimental result in which cellular behavior is regulated by control has been obtained. This result suggests that control methods of GRNs can be realized. Motivated by the above backgrounds, we study a control method of GRNs.

GRNs are in general modeled by ordinary/partial differential equations with high nonlinearity and high dimensionality. In order to deal with such a system, it is important to consider a simple model, and various models such as Bayesian networks, Boolean networks (BNs) [3], hybrid systems (piecewise affine models), and Petri nets have been developed so far (see, e.g., [4] for further details). In control problems, BNs and hybrid systems are frequently used [5–10]. In the hybrid systems-based approach, the classes of GRNs are limited to low-dimensional systems, because the computation time to solve the control problem is too long. In a BN, gene expression is expressed by a binary value (ON or OFF), and dynamics such as interactions between genes are expressed by Boolean functions [3]. In, for example, [11], it is pointed out that a BN is too simple as a model of GRNs. However, there is an advantage that a BN can be relatively applied to large-scale systems. Furthermore, since the behavior of GRNs is stochastic by the effects of noise, it is appropriate that a Boolean function is randomly decided at each time among the candidates of Boolean functions. Thus, a probabilistic Boolean network (PBN) has been proposed in [12], and further, a context-sensitive PBN (CS-PBN) has been proposed as a general form of PBNs [13, 14]. In a CS-PBN, the deciding time is also randomly selected.

In this paper, a CS-PBN is adopted as a model of GRNs, and for CS-PBNs, the verification problem and the optimal control problem are considered. In the verification of PBNs, a solution method using the probabilistic model checker PRISM [15] has been proposed in [16]. However, this PRISM-based method for PBNs has not been extended to that for CS-PBNs. In optimal control of PBNs and CS-PBNs, many results have been obtained so far (see, e.g., [13, 14, 17–24]). In many existing results, state transition diagrams with nodes (i.e., transition probability matrices) must be computed for a (CS-)PBN with genes. In order to compute state transition diagrams, several issues such as memory consumption must be considered in implementation, and it is desirable to directly use a given Boolean function. The authors have proposed in [22, 23] control methods in which state transition diagrams are not computed. Comparing the methods in [22, 23] with other existing results [13, 14, 17–21, 24], the methods in [22, 23] can relatively handle more large-scale GRNs. The method in [22] can be applied to PBNs and CS-PBNs, but the expected value of a given nonnegative function cannot be evaluated as a cost function (objective function). In the method in [23], the expected value of a given nonnegative function can be used as a cost function, and the optimal control problem is reduced to a polynomial optimization problem. However, this method has been proposed for PBNs, and an extension to CS-PBNs has not been discussed so far. Thus, for verification of CS-PBNs, the PRISM-based method for PBNs [16] is extended to that for CS-PBNs. For optimal control of CS-PBNs, a solution method using polynomial optimization [23] is extended to that for CS-PBNs. Furthermore, the effectiveness of the proposed methods is presented by a numerical example on the WNT5A network, which is related to melanoma. The proposed methods provide us useful tools in control theory of GRNs.

This paper is organized as follows. In Section 2.1, a CS-PBN is explained. In Section 2.2, a solution method for the verification problem is proposed. In Section 2.3, a solution method for the optimal problem is proposed. In Section 3, a numerical example is presented. In Section 4, we conclude this paper.

Notation. Let denote the set of real numbers. Let denote the set of -dimensional vectors, which consists of elements and . For a matrix , denotes the transpose of .

2. Materials and Methods

2.1. Context-Sensitive Probabilistic Boolean Networks

First, we introduce a probabilistic Boolean network (PBN). Consider the following PBN:

where is the state (e.g., the expression of genes), is the control input (e.g., the expression of genes), that is, the value of can be arbitrarily given, and is the discrete time. For a fixed , is a given Boolean function consisting of logical operators such as AND (), OR (), and NOT (). In deterministic Boolean networks, is uniquely determined for given , , and . In PBNs, the candidates of are given, and for each , selecting one Boolean function is probabilistically independent at each time. Let , , denote the candidates of . The probability that is selected is defined by

Then, the following relation:

must be satisfied. Probabilistic distributions are derived from experimental results, but details are one of the future works. Then, a method for inferring a probabilistic Boolean network will be useful (see, e.g., [25]).

Example 1. As a simple example, consider the following deterministic Boolean network of an apoptosis network [26, 27] (see also Figure 1):

where the concentration level (high or low) of the inhibitor of apoptosis proteins (IAP) is denoted by , the concentration level of the active caspase 3 (C3a) by , and the concentration level of the active caspase 8 (C8a) by . The concentration level of the tumor necrosis factor (TNF, a stimulus) is denoted by and is regarded as the control input. Although Boolean dynamics in the above system are synchronous, both synchronous and asynchronous dynamics will be included. From this viewpoint, we consider the following PBN induced by the above system:

where , and we give satisfying . In addition, all state trajectories can be expressed as the state transition diagram with nodes.

Figure 1: Simplified model of an apoptosis network. Activation (solid), inhibition (broken).

In PBNs, we suppose that selecting one Boolean function is probabilistically independent at each time. However, it will be natural to consider the situation that switches of Boolean functions do not occur frequently. From this viewpoint, a context-sensitive PBN (CS-PBN) has been proposed in [13, 14]. In CS-PBNs, the deciding time of Boolean functions is also selected randomly. Hereafter, let denote the probability that Boolean functions are switched at time , and a pair of the system (1) and is called a CS-PBN.

To compare CS-PBNs with PBNs, consider (7) as a simple example. In PBNs, a switch of and functions does not depend on the Boolean function at time . In CS-PBNs, a switch of and is decided by the discrete-time Markov chain in Figure 2. In other words, this switch depends on the Boolean function at time . Owing to this difference, a control/verification method for CS-PBNs cannot be directly derived from that for PBNs.

Figure 2: Discrete-time Markov chain in .

2.2. Verification Using Model Checking

First, the reachability problem is formulated as the verification problem studied in this paper. The reachability problem is one of the typical verification problems. For a given CS-PBN, the output is defined, where , , . We remark that the output does not mean the measured signal. First, the reachability problem is formulated as follows.

Problem 2 (reachability problem). Suppose that, for CS-PBN with the output, the initial state , the initial Boolean function (), the control time , and the target output are given ( is not given). Then, find a maximum probability that holds within time by manipulating a control input sequence .

In the standard reachability problem, only terminal time is focused, and it is checked whether holds or not. In this paper, we focus on not only terminal time but also other times . Furthermore, since a CS-PBN has the control input, which can be regarded as a nondeterministic variable, we find a maximum probability satisfying the condition.

Next, we will propose a solution method for Problem 2. As a preparation, the following lemma [28] is introduced.

Lemma 3. Consider two binary variables . Then, the following relations hold.

(i) is equivalent to .(ii) is equivalent to .(iii) is equivalent to .

For example, is equivalently transformed into . By using this lemma, a Boolean function can be transformed into a polynomial on the real number field.

To solve Problem 2, the probabilistic model checker PRISM [15] is used. PRISM supports a discrete-time Markov chain (DT-MC), a continuous-time Markov chain (CT-MC), and a Markov decision process (MDP). PRISM consists of three parts: “Model,” “Properties,” and “Simulator.” In the “Model” part, a given probabilistic system is described using the PRISM language. In the “Properties” part, the property specification language incorporates temporal logic such as PCTL (probabilistic computation tree logic) [29], and we can verify if a given PCTL formula holds. In the “Simulator,” the state trajectories can be simulated.

Now, using PRISM, we propose a method for modeling a given CS-PBN. By modeling a given CS-PBN via PRISM, Problem 2 can be solved. In the PRISM-based method, Boolean functions in a given PBN can be directly used. To explain the PRISM-based method, consider the PBN (5)–(7) in Example 1 and . Suppose that the initial state and the initial Boolean function are given by , , , and (i.e., ), respectively. By using Lemma 3, each Boolean function can be transformed into some polynomial on the field of real numbers. Then, the PRISM code describing this CS-PBN is shown in Figure 3.

Figure 3: PRISM code expressing the CS-PBN.

In line 1, it is described that a given system is a MDP; that is, the control input (in other words, the nondeterministic variable) that must decide is included. In line 2, the probability is given by . In lines 3–7, the discrete-time Markov chain such as Figure 2 is modeled for . The probabilistic variable d1 corresponds to in . In line 4, is given by . In lines 5-6, the behavior of d1 is modeled. In line 5, it is described that if holds, then the next state is with the probability and with the probability . In lines 8–12, is modeled. In line 9, it is described that takes a binary value, and the initial value of is given by . In line 10, is modeled. In line 11, is modeled. In a similar way, is modeled in lines 13–22, and is modeled in lines 23–32. In CS-PBNs, a discrete probabilistic distribution is given for each . Hence, , , must be modeled separately. To associate with each module, [CSPBN] is described. Finally, in lines 33–39, the property of the control input is described as a nondeterministic variable. Note that the initial value of the control input must be given (see line 34). Hence, PRISM must be executed for two cases of and .

The above explanation is the outline of the PRISM-based modeling method. Based on the above example, we propose a procedure for deriving the PRISM code expressing a general CS-PBN.

2.2.1. Procedure for Modeling CS-PBNs

Step 1. Transform each Boolean function into a polynomial on the real number field by using Lemma 3. The obtained Boolean functions are denoted by .

Step 2. Describe that a given system is a MDP, and give .

Step 3. Describe modules CSPBNm and CSPBN , , as follows:

 module CSPBNm

   : init ; [CSPBN] ; [CSPBN] ;  [CSPBN] ;

 endmodule module CSPBN

  init ; [CSPBN] (); [CSPBN] ();  [CSPBN] ();

 endmodule

Step 4. Describe the control input , , as follows:

 module input

  init ; [PBN1]  [PBN1]  [PBN1]  [PBN1]

 endmodule

Finally, consider solving Problem 2. For solving this problem, we use prepared in PRISM. For example, suppose that and . Then, in PRISM, Problem 2 can be described by

Therefore, we see that Problem 2 can be solved using PRISM. The control input sequence is obtained simultaneously, but in PRISM 4.0.3, the obtained control input sequence cannot be displayed except for the case of . In the case of , the discrete-time Markov chain can be obtained as the closed-loop system of a given CS-PBN.

2.3. Optimal Control Using Polynomial Optimization

Consider the following problem.

Problem 4 (optimal control problem). Suppose that, for CS-PBN, the initial state , the initial Boolean function , and the control time are given ( is not given). Then, find a control input sequence minimizing the cost function

where , are weighting vectors whose element is a nonnegative real number and denotes a conditional expected value.

According to the following two reasons, the linear cost function (9) is appropriate. (i) For a binary variable , the relation holds. That is, in the cost function, the quadratic term such as is not necessary. (ii) In control of GRNs, expression of a certain gene is frequently focused (see, e.g., [19]). That is, in the cost function, the quadratic term such as , , is not necessary.

For a PBN, the authors have derived the following recursive representation of the expected value of the state:

where the condition in the expected value is omitted. See [23] for further details. In this paper, this representation is extended to that in CS-PBNs.

First, we present a simple example. Consider the PBN (5)–(7) in Example 1. Suppose that the initial state and the initial Boolean function are given by , , , and , respectively.

Consider deriving the expected value of the state at time . Suppose that . Since the Boolean function at time is given, the state at time is uniquely derived as

Hereafter, the condition such as , in the expected value is omitted. Next, consider deriving the expected value of the state at time . We remark that, for each , the discrete-time Markov chain such as Figure 2 can be obtained. For example, the probability that is selected at time is , and the probability that is selected at time is . Suppose that . Then, we can obtain

Finally, we remark that the probability that some Boolean function is selected is time-varying. For example, the probability that is selected at time is , and the probability that is selected at time is .

Next, consider a general case. From the observation of the above example, we can obtain the following recursive representation:

where and

Therefore, Problem 4 can be reduced to the following polynomial optimization problem:

The constraint guarantees that is a binary variable. A polynomial optimization problem can be solved by using a suitable solver such as SparsePOP [30].

3. Results and Discussion

In this section, we present a numerical example on the WNT5A network. First, the WNT5A network is explained. Next, computational results are presented.

3.1. WNT5A Network

Consider the GRN with the gene WNT5A, which is related to melanoma. A Boolean network model is given by

where the concentration level (high or low) of the gene WNT5A is denoted by , the concentration level of the gene pirin by , the concentration level of the gene S100P by , the concentration level of the gene RET1 by , the concentration level of the gene MART1 by , the concentration level of the gene HADHB by , and the concentration level of the gene STC2 by . See [31] for further details.

Next, suppose that the control input is given by (the concentration level of the gene pirin), according to the discussion in [19]. By replacing and with and , respectively, we can obtain the following model:

Furthermore, we add the probabilistic behavior as follows:

where holds. Thus, we can obtain the PBN model expressing a WNT5A network.

3.2. Computational Result on Verification

Consider solving Problem 4. For the PBN (18), we assume that and are given by and , respectively. In the WNT5A network, it is important to inhibit the concentration level of the gene WNT5A [32]. From this fact, we set and . The initial state is given by . The initial Boolean function is given by . In addition, we set .

Next, we show the computation result. Then, we can obtain for , for , and for . It is desirable that is close to . Hence, we see that the performance is degraded for a larger .

3.3. Computational Result on Optimal Control

Consider solving Problem 4. For the PBN (18), we assume that and are given by and , respectively. Since the concentration level must be inhibited, the weights , and in Problem 4 are given by

respectively. The initial state is given by . The initial Boolean function is given by . In addition, we set and .

Next, we show the computation result. By solving Problem 4, we can obtain , . The expected value of the state at each time is obtained as

Hence, we see that the concentration level of the gene WNT5A is inhibited with time.

In addition, the optimal value of the cost function was 5.23. For and , we can obtain and , respectively. From these values, we see that the performance is degraded for a larger .

4. Conclusions

In this paper, we discussed verification and optimal control for a context-sensitive probabilistic Boolean network (CS-PBN), which is one of the models for gene regulatory networks (GRNs). In verification, the PRISM-based method for PBNs [16] was extended to that for CS-PBNs. In optimal control, the optimal control method for PBNs [23] was extended to that for CS-PBNs. A CS-PBN is a generalized version of a PBN, and it enables us to consider several situations. Furthermore, as a numerical example, we considered the WNT5A network, which is related to melanoma. The proposed methods provide us useful tools in control theory of GRNs.

In recent years, a stochastic Boolean network [33] has been proposed as a new representation of PBNs. In addition, to simplify a given Boolean network, the Karnaugh map realization of a Boolean network has been proposed in [34]. These modeling methods will be useful for reducing the computational burden. Future efforts will focus on applying these modeling methods to the control problem and the verification for CS-PBNs.

Conflict of Interests

The authors declare that they have no conflict of interests regarding the publication of this paper.

Acknowledgment

This work was partially supported by Grant-in-Aid for Young Scientists (B) 23760387.

References

H. Santos-Rosa and C. Caldas, “Chromatin modifier enzymes, the histone code and cancer,” European Journal of Cancer, vol. 41, no. 16, pp. 2381–2402, 2005.

A. Milias-Argeitis, S. Summers, J. Stewart-Ornstein et al., “In silico feedback for in vivo regulation of a gene expression circuit,” Nature Biotechnology, vol. 29, no. 12, pp. 1114–1116, 2011.

S. A. Kauffman, “Metabolic stability and epigenesis in randomly constructed genetic nets,” Journal of Theoretical Biology, vol. 22, no. 3, pp. 437–467, 1969.

H. de Jong, “Modeling and simulation of genetic regulatory systems: a literature review,” Journal of Computational Biology, vol. 9, no. 1, pp. 67–103, 2002.

T. Akutsu, M. Hayashida, W.-K. Ching, and M. K. Ng, “Control of Boolean networks: Hardness results and algorithms for tree structured networks,” Journal of Theoretical Biology, vol. 244, no. 4, pp. 670–679, 2007.

S.-I. Azuma, E. Yanagisawa, and J.-I. Imura, “Controllability analysis of biosystems based on piecewise-affine systems approach,” IEEE Transactions on Automatic Control, vol. 53, no. 1, pp. 139–152, 2008.

K. Kobayashi, J.-I. Imura, and K. Hiraishi, “Polynomial-time algorithm for controllability test of a class of Boolean biological networks,” Eurasip Journal on Bioinformatics and Systems Biology, vol. 2010, Article ID 210685, 12 pages, 2010.

K. Kobayashi and K. Hiraishi, “Optimal control of Boolean biological networks modeled by Petri nets,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. E96-A, no. 2, pp. 532–539, 2013.

K. Kobayashi and K. Hiraishi, “Optimal control of gene regulatory networks with effectiveness of multiple drugs: a Boolean network approach,” BioMed Research International, vol. 2013, Article ID 246761, 11 pages, 2013.

C. J. Langmead and S. K. Jha, “Symbolic approaches for finding control strategies in boolean networks,” Journal of Bioinformatics and Computational Biology, vol. 7, no. 2, pp. 323–338, 2009.

A. Mochizuki, “An analytical study of the number of steady states in gene regulatory networks,” Journal of Theoretical Biology, vol. 236, no. 3, pp. 291–310, 2005.

I. Shmulevich, E. R. Dougherty, S. Kim, and W. Zhang, “Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks,” Bioinformatics, vol. 18, no. 2, pp. 261–274, 2002.

B. Faryabi, G. Vahedi, J.-F. Chamberland, A. Datta, and E. R. Dougherty, “Intervention in context-sensitive probabilistic boolean networks revisited,” Eurasip Journal on Bioinformatics and Systems Biology, vol. 2009, Article ID 360864, 13 pages, 2009.

R. Pal, A. Datta, M. L. Bittner, and E. R. Dougherty, “Intervention in context-sensitive probabilistic Boolean networks,” Bioinformatics, vol. 21, no. 7, pp. 1211–1218, 2005.

M. Kwiatkowska, G. Norman, and D. Parker, “PRISM 4.0: verification of probabilistic real-time systems,” in Proceedings of the 23rd International Conference on Computer Aided Verification, vol. 6806 of Lecture Notes in Computer Science, pp. 585–591, Springer, 2011.

K. Kobayashi and K. Hiraishi, “Symbolic approach to verification and control of deterministic/probabilistic Boolean networks,” IET Systems Biology, vol. 6, no. 6, pp. 215–222, 2012.

X. Chen, T. Akutsu, T. Tamura, and W. K. Ching, “Finding optimal control policy in probabilistic Boolean Networks with hard constraintsby using integer programming and dynamic programming,” International Journal of Data Mining and Bioinformatics, vol. 7, no. 3, pp. 321–343, 2013.

W.-K. Ching, S.-Q. Zhang, Y. Jiao, T. Akutsu, N.-K. Tsing, and A. S. Wong, “Optimal control policy for probabilistic Boolean networks with hard constraints,” IET Systems Biology, vol. 3, no. 2, pp. 90–99, 2009.

A. Datta, A. Choudhary, M. L. Bittner, and E. R. Dougherty, “External control in Markovian genetic regulatory networks,” Machine Learning, vol. 52, no. 1-2, pp. 169–191, 2003.

A. Datta, A. Choudhary, M. L. Bittner, and E. R. Dougherty, “External control in Markovian genetic regulatory networks: the imperfect information case,” Bioinformatics, vol. 20, no. 6, pp. 924–930, 2004.

S. Z. Denic, B. Vasic, C. D. Charalambous, and R. Palanivelu, “Robust control of uncertain context-sensitive probabilistic Boolean networks,” IET Systems Biology, vol. 3, no. 4, pp. 279–295, 2009.

K. Kobayashi and K. Hiraishi, “An integer programming approach to optimal control problems in context-sensitive probabilistic Boolean networks,” Automatica, vol. 47, no. 6, pp. 1260–1264, 2011.

K. Kobayashi and K. Hiraishi, “Optimal control of probabilistic Boolean networks using polynomial optimization,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. E95-A, no. 9, pp. 1512–1517, 2012.

R. Pal, A. Datta, and E. R. Dougherty, “Optimal infinite-horizon control for probabilistic Boolean networks,” IEEE Transactions on Signal Processing, vol. 54, no. 6, pp. 2375–2387, 2006.

S. Marshall, L. Yu, Y. Xiao, and E. R. Dougherty, “Inference of a probabilistic boolean network from a single observed temporal sequence,” Eurasip Journal on Bioinformatics and Systems Biology, vol. 2007, Article ID 32454, 15 pages, 2007.

M. Chaves, “Methods for qualitative analysis of genetic networks,” in Proceedings of the European Control Conference, pp. 671–676, 2009.

L. Tournier and M. Chaves, “Uncovering operational interactions in genetic networks using asynchronous Boolean dynamics,” Journal of Theoretical Biology, vol. 260, no. 2, pp. 196–209, 2009.

H. P. Williams, Model Building in Mathematical Programming, John Willey & Sons, 4th edition, 1999.

F. Ciesinski and M. Groesser, “On probabilistic computation tree logic,” in Validation of Stochastic Systems, vol. 2925 of Lecture Notes in Computer Science, pp. 147–188, 2004.

SparsePOP, http://www.is.titech.ac.jp/~kojima/SparsePOP/SparsePOP.html

Y. Xiao and E. R. Dougherty, “The impact of function perturbations in Boolean networks,” Bioinformatics, vol. 23, no. 10, pp. 1265–1273, 2007.

A. T. Weeraratna, Y. Jiang, G. Hostetter et al., “Wnt5a signaling directly affects cell motility and invasion of metastatic melanoma,” Cancer Cell, vol. 1, no. 3, pp. 279–288, 2002.

J. Liang and J. Han, “Stochastic Boolean networks: an efficient approach to modeling gene regulatory networks,” BMC Systems Biology, vol. 6, article 113, 2012.

R. K. Layek, A. Datta, and E. R. Dougherty, “From biological pathways to regulatory networks,” Molecular BioSystems, vol. 7, no. 3, pp. 843–851, 2011.