Modeling and Analysis of the Obstacle-Avoidance Strategies for a Mobile Robot in a Dynamic Environment

Obstacle avoidance is a key performance of mobile robots. However, its experimental verification is rather difficult, due to the probabilistic behaviors of both the robots and the obstacles. This paper presents the Markov Decision Process based probabilistic formal models for three obstacle-avoidance strategies of a mobile robot in an uncertain dynamic environment. The models are employed to make analyses in PRISM, and the correctness of the analysis results is verified by MATLAB simulations. Finally, the minimum time and the energy consumption are determined by further analyses in PRISM, which prove to be useful in finding the optimal strategy. The present work provides a foundation for the probabilistic formal verification of more complicated obstacleavoidance strategies.


Introduction
Obstacle avoidance is a key technique for the design and applications of mobile robots, because static barriers and even dynamic obstacles frequently exist in their paths.When several robots move in the same region, they act in fact as obstacles to one another themselves [1,2].Therefore, studies on obstacle avoidance have become an active topic in the field of mobile robots, and a series of strategies have been proposed to realize the avoidance of static and/or dynamic obstacles [2][3][4][5].In the above references, the dynamic environments of the mobile robots are known in advance, because the obstacles are assumed to have predefined moving behaviors.However, mobile robots occasionally have to work in uncertain circumstances, where the movements of the obstacles cannot be predicted accurately [6,7].Besides, the sensor and actuator noises, the time delay of network, and other factors may still lead to inaccurate operations of the robots themselves [8].Due to these reasons, probability-based obstacle-avoidance algorithms have been put forward in recent years [9][10][11], which models both the movements of the obstacles and the operations of the robots as probabilistic events.
The correctness of obstacle-avoidance algorithm is very important.Testing has been the most frequently used method in the performance validation of robots, but it is by no means the best way [12].Its deficiencies mainly lie in two aspects: (1) the testing results are incomplete, because only a subset of all the possible cases can be examined by physical experiments or software simulations; (2) the testing results are generally small-sample data that are unsuitable for probabilistic analysis.
Formal verification [13] is a useful alternative method to physical experiments and software simulations, because it is not only complete in logic but also adaptable for the description and analysis of probabilistic events [14][15][16].For example, probabilistic model checking (PMC) is a probability-based formal verification method widely used in the performance validation of software and hardware [17,18].There are three main steps involved in PMC.First, a mathematical model is built to incorporate all the possible probabilistic behaviors; second, Probabilistic Computation Tree Logic (PCTL) formulae [19] are derived to describe the key logical requirements; third, a software tool such as PRISM [20] is employed to check whether the mathematical model satisfies the logical requirements or not.If all the requirements are fully satisfied, the probabilistic behaviors are validated.Otherwise, it implies that some errors may exist in the original model.In recent years, formal verification has 2 Mathematical Problems in Engineering been widely used to verify the path planning of mobile robots.However, PMC has been scarcely applied in validating the obstacle-avoidance strategies.
The purpose of the present paper is to verify three avoidance algorithms that involve a mobile robot and a single dynamic obstacle [9].The paths of the robot and obstacle cross each other, and their movements have the properties of Markov Decision Processes (MDPs) [21,22].The probabilistic models and the PCTL formulae are first built, and then the PRISM software is employed to make analyses on the three avoidance strategies.Both the time and the energy requirements for the robot to reach the destination have been obtained, and the time or energy based optimal condition is thereby applied to help find the optimal obstacle-avoidance strategy under diverse circumstances.In addition, simulations are also made in MATLAB to verify the correctness of the results obtained by PRISM.To this end, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB.
The rest of the paper is organized as follows.Section 2 introduces the necessary preliminaries.In Section 3, we formulate the obstacle-avoidance algorithms.Then, the MDPs are constructed in Section 4, and the verification on the algorithm is made in Section 5. Finally, conclusions are presented in Section 6.

Markov Decision Process.
A Markov Decision Process (MDP) is generally defined as a multituple [18].In the present work, a five-tuple, (,  0 , Acts, Steps, Cost), is adopted to describe the probabilistic model, where (1)  represents a group of states, (2)  0 is the initial state and it belongs to , (3) Acts stand for a set of actions, each of which has its own probability, (4) Steps are the state transition functions, (5) Cost is a reward function that describes quantitative information of the model.

Probabilistic Computation Tree
Logic.Probabilistic Computation Tree Logic (PCTL) is an extension of the nonprobabilistic Computation Tree Logic (CTL), which expresses the qualitative properties of the probabilistic models [19].Generally, a PCTL is expressed as a formula that connects the properties of the MDP using the Boolean operators (such as ¬ (negation) and ∧ (conjunction)), the temporal operators (such as  (next), U (until),  (future), and  (globally)), the probabilistic operator , and a reward (or cost) operator .
A PCTL formula can be used to calculate a certain quantity of the MDP such as the maximum probability and the minimum time (or energy) consumption.For example, using the following formula one can compute the maximum probability of a state that satisfies both the two specifications spec1 and spec2.Again, the following formula can be used to find the minimum cost of energy when the specification spec3 is satisfied: Finally, consider another PCTL formula Obviously, it can be used to calculate the maximum probability of a state, which meets the two requirements spec4 and spec6 but does not satisfy spec5.

Obstacle-Avoidance Algorithms
3.1.Probabilistic Model of a Dynamic Obstacle.Divide the whole moving process of the dynamic obstacle into  steps, each of which has the same time interval Δ.Assume that the dynamic obstacle has a stepwise uniform motion; that is, it has different velocity at different time step.In the whole process of movement, its minimum and maximum velocities are denoted by  min and  max .Then, the probability density The paths of a mobile robot and a dynamic obstacle.
function (; ) for the obstacle to reach the position  after  steps can be expressed as [9]  ( where Here,  0 ,  2 0 , and  are the initial position, variance, and average velocity, respectively.Integrating (; ) along the practical path of the obstacle, one can obtain the probability of reaching the position .

Obstacle-Avoidance Strategies of a Mobile Robot.
Here, three obstacle-avoidance strategies are given for a mobile robot with only a single dynamic obstacle.As shown in Figure 2, the former and the latter are represented by a circle and a square, respectively.
Assume that the paths of the robot and the obstacle intersect at region  with angle  (0 ∘ <  < 180 ∘ ) (see Figure 2). 1 and  1 represent their locations when they begin to enter region , while  2 and  2 stand for those when they just leave such a region completely. 0 is a reference position of the robot, which can be specified at an arbitrary point not over  1 . 0 and  3 are two reference positions of the obstacle to be determined in subsequent derivation. 3 is an undetermined position of the robot.
Under this condition, we mainly consider three avoidance strategies: Acceleration, Deceleration, and Detour.In the Acceleration mode, the robot goes across region  earlier than the obstacle does by enhancing its velocity; in the Deceleration one, it passes region  later than the obstacle does by decreasing its velocity; in the Detour one, it realizes obstacle avoidance by changing its moving direction as shown in Figure 3, where  is the rotation angle and  is the rotation radius.When  and  are given, the robot may go along either  0  4 or  0  5 , due to its probabilistic behavior.Here,  0  4 represents the expected Detour path, while  0  5 denotes the unexpected one.In Figure 3,  4 represents the position where the obstacle begins to enter the path intersection region in the expected Detour mode.
However, how to choose one of these three strategies under a specific circumstance is still a problem that needs to be solved.Here, the concept of Dangerous Region is introduced to help explain the decision-making process of the robot.If the obstacle stays in such a region at the initial time, the robot has to choose a certain strategy to avoid collision.
To describe such a concept, one needs to make the following three parameters clear at first.(a) Reference position  0 : when the two paths are given, the position of  1 is fixed (see Figure 2).Therefore, the point  0 can be determined by the distance  in , which is defined as [9] where   is the initial velocity of the robot and   is the distance between  1 and  2 .
(c) Reference position  3 : it is defined in such a way that the obstacle can take less time than   to go from it to  2 with the velocity  min .
Then, we name the region between  0 and  3 as the Dangerous Region.Note: the distance between  0 and  3 is less than that between  0 and  2 .

Construction of MDPs
This section gives the MDP models for both the obstacle and the robot.The model symbols are shown in Symbol Description.the locations  0 ,  1 ,  3 ,  2 , and  4 in turn.Under this condition, its movement is divided into seven states, and then its formal model is constructed as shown in Figure 4.

Formal Model of the
According to the obstacle-avoidance strategies,  0  3 is the Dangerous Region.Therefore, if the obstacle is in the state  0  1 or  1  3 , the robot should choose a strategy to avoid collision.
To help understand the model, three typical state transitions are explained as follows: (1) ⟨O  O 0 When the timer  is less than    0 , the obstacle is in the state    0 ; that is, the obstacle is moving in the region    0 .
(2) ⟨   0 When  becomes equal to    0 , the obstacle switches to the state  0  1 from the state    0 ; that is, the obstacle moves into the region  0  1 from the region    0 , and then the timer  is reset to 0.
The obstacle stays in the state     and will not switch to other states; that is, the obstacle reaches its terminal position.
Dividing each region in Figure 4 into some equal steps in time, one can compute the probability for the obstacle to reach the position  by integrating the probability density function (; ) in (5).

Formal Model of the Mobile Robot.
This section provides the formal representation for the state transitions of the robot (see Figures 5-7).In the description, we assume that each avoidance strategy can be executed by the robot with a certain probability.Specifically, when the robot takes the "Detour" operation, it can rotate to the expected angle with probability p1, as a result of the wheel slippage and other factors; if the robot selects the action "Acceleration" or "Deceleration," the probability of changing its velocity to the expected value is 2 or 3, because of the actuator error and other uncertainties.

Detour Strategy.
The mobile robot moves from its initial position   to the reference one  0 and then takes the detour strategy to avoid the obstacle.If the robot rotates to the expected angle, it can move into the desired region  0  4 successfully (see Figure 3).Otherwise, it may go into another arbitrary region such as  0  5 (see Figure 3).As stated above, it rotates to the expected angle with probability 1.Therefore, the robot can go into  0  4 with probability 1, while it will deviate into  0  5 with probability 1 − 1.In the former case, the collision will be effectively avoided and the robot can go to its terminal   safely, because the detour operation is successful.However, in the latter case, whether the robot will collide with the obstacle or not is still uncertain.If the collision is avoided, the robot will also go to   successfully.Otherwise, the collision will make the detour operation fail.
To help understand the model, four typical state transitions are explained as follows: (1) When the mobile robot is in the state    0 and the value of Safe is equal to 0 (i.e., the obstacle is in the Dangerous Region), the robot will take the detour strategy to switch to the state  0  4 .The detour operation can be successfully made with probability 1. ( When the robot is in the state    0 and the value of Safe is equal to 0, the robot will switch to the state  0  5 with probability (1 − 1).
(3) ⟨ 0  5 When the robot is in the state  0  5 and the value of Conflict is equal to 1 (i.e., the robot collides with the obstacle), the robot will switch to the Collision state.The robot will stay in the Collision state and stop moving.

Acceleration Strategy.
If the robot locating at  0 takes the Acceleration strategy to avoid collision, it will move to  2 by enhancing its velocity.As stated above, the probability of increasing its velocity to the expected value is 2.Therefore, it can reach  2 with probability 2 and then go to the terminal   safely.In this strategy, it is easy to infer that the change of velocity will fail with probability 1 − 2.When the desired velocity is not obtained, whether the robot will collide with the obstacle or not is uncertain.If the collision is avoided, the robot will also go to   successfully.
Two typical state transitions are explained as follows: (1) When the robot is in the state    0 and the value of Safe is equal to 0, the robot chooses the Acceleration action.Its state switches from    0 to  0  2 with probability 2.
When the robot is in the state    0 and the value of Safe is equal to 0, the robot chooses the Acceleration action.Its state will change from    0 to  0  2 with probability (1− 2), because the Acceleration action will fail with probability (1 − 2).

Deceleration Strategy.
If the robot locating at  0 takes the Deceleration strategy to avoid collision, it will move to a certain position  5 between  0 and  1 by reducing its velocity.As stated above, the probability of decreasing its velocity to the expected value is 3.Therefore, it can reach  5 with probability 3 and then go to the terminal   safely.Obviously, the change of velocity will fail with probability 1−3.When the desired velocity is not obtained, whether the robot will collide with the obstacle or not is uncertain.If the collision is avoided, the robot will also go to   successfully.
Two typical state transitions are as follows: (1) When the robot is in the state    0 and the value of Safe is equal to 0, the robot chooses the Deceleration action.Its state switches from    0 to  0  3 with probability 3.( When the robot is in the state    0 and the value of Safe is equal to 0, the robot chooses the Deceleration action.Its state will change from    0 to  0  3 with probability (1− 3), because the Deceleration action will fail with probability (1 − 3).

Probabilistic Analysis and Verification
In this section, we take an example to make probabilistic analysis and verification.The velocities and geometrical parameters of the example are shown in Figure 8, where   and   represent the diameter of the circle and the side length of the square, respectively.
Next, probabilistic analysis is performed in the PRISM software to get the probability-time curves of the three obstacle-avoidance strategies.Then, the corresponding computation is also carried out in MATLAB to verify the effectiveness of the analysis in PRISM.On this basis, the time and energy consumption of the three strategies are calculated in PRISM to help find the optimal strategy.

Mobile Robot Transiting through the Expected States.
Here, we consider the cases of the mobile robot moving through the states  0  3 in Detour,  0  2 in Acceleration, and  0  5 in Deceleration.It is assumed that 1 = 2 = 3 = 1.

Probability-Time Curves Given by PRISM.
To get the probability-time curves by analysis in PRISM, one has to construct the following PCTL formula at first: where the value 1 of Success means that the robot succeeds in avoiding the obstacle.The analysis results corresponding to the three avoidance strategies are shown in Figure 9.
(1) Detour Strategy.(The rotation radius  of the robot is prescribed as 1.0 cm.)If the time for the robot to pass the region  0  4 is less than or equal to 7 s, the obstacle will be successfully avoided.
(2) Acceleration Strategy.If the time to go across the region  0  2 does not exceed 4 s, the robot is able to avoid collision.
(3) Deceleration Strategy.If the time to pass the region  0  1 is more than or equal to 21 seconds, the collision can be prevented.

Simulation Results by MATLAB.
In this subsection, MATLAB is used to make simulation to verify the correctness of the results obtained by PRISM.To this end, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB.
Nevertheless, how to determine the input time is still a problem needing consideration.In the strategy of Detour or Acceleration, if the robot takes longer time to pass the region  0  4 or  0  2 , the probability for it to avoid collision will become smaller.However, shorter time to pass  0  1 will give rise to smaller obstacle-avoidance probability in the Deceleration mode.Therefore, we choose the maximum time of obstacle avoidance in Detour (i.e., 7 s) or Acceleration (i.e., 4 s) but the minimum one in Deceleration (i.e., 21 s) to make verification in MATLAB.
The visualized results of the simulation are provided in Figure 10.It is seen that the collision is effectively avoided in each case, which verifies the analysis results in PRISM.

Mobile Robot Transiting through the Unexpected States.
In this subsection, we consider the cases of the mobile robot moving through the states  0  5 in Detour,  0  2 in Acceleration, and  0  3 in Deceleration.

Verification Results by PRISM. The PCTL formula is
where the value 1 of ErrorSuccess means that the robot succeeds in avoiding the obstacle finally, even if it goes through the unexpected state.The probability-time curves of the three obstacleavoidance strategies are shown in Figure 11.
(1) Detour Strategy.Assume that the expected rotation angle is 45 ∘ , but the actual one is only 15 ∘ or 30 ∘ .Under this condition, the robot will adjust its motion to avoid collisions.Figure 11(a) shows that when the actual rotation angle is 15 ∘ or 30 ∘ the maximum time to successfully avoid collision is 5 s or 6 s, respectively.
(2) Acceleration.Like in case , if the time for the robot to reach  2 is less than or equal to 4 s, the obstacle will be avoided (see Figure 11(b)).
(3) Deceleration Strategy.If the time to arrive at  1 is longer than 7 s, the robot can avoid the obstacle (see Figure 11(c)).

Simulation Results by MATLAB.
In this subsection, simulations are carried out in MATLAB to verify the analysis results shown in Figure 11.
(1) Detour Strategy.Here, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB.Therefore, when the actual rotation angle is 15 ∘ or 30 ∘ , we take 5 s or 6 s to perform simulation.Figure 12 shows the visualized results, which obviously verify the results in Figure 11(a).
(2) Acceleration or Deceleration Strategy.For Acceleration or Deceleration, we take six different values of time and four different values of 2 or 3 to make simulations in MATLAB.In each case, 100 times of simulations are performed to calculate the probability of obstacle avoidance.The results are given in Tables 1 and 2, which are consistent with those shown in Figures 11(b) and 11(c).From this, we come to the conclusion that probabilistic model checking is an effective method to verify the obstacle-avoidance strategies of mobile robots.from its initial position to the destination are used to find the optimal obstacle-avoidance strategy.For this purpose,  The PCTL formula corresponding to the minimum time is

Minimum Time and Energy
where the value 1 of Done means the robot reaches the destination safely.
The PCTL formula associated with the minimum energy is In addition, the kinetic energy of the robot can be calculated as follows: where  is the mass,  is the rotational inertia, V is the linear velocity, and  is the angular velocity.The analysis results of the minimum time and energy consumption are shown in Table 3.In the Detour mode, if actual rotation angle is 45 ∘ , the robot arrives at its destination by passing the expected state; otherwise, it reaches the terminal by going through the unexpected state.It is seen that the expected path costs the lowest "minimum energy" but the longest "minimum time," while the unexpected trajectory (15 ∘ ) consumes the shortest "minimum time" but the highest "minimum energy."In the acceleration mode, the expected path ( 0  2 = 4) requires the shortest "minimum time" but the highest "minimum energy," while the unexpected one ( 0  2 = 6) needs the lowest "minimum energy" but the longest "minimum time."In the Deceleration mode, the expected path ( 0  3 = 21) spends the longest "minimum time," while all the paths expend the same "minimum energy."Further comparison shows that the Acceleration strategy ( 0  2 = 4) is the least time-costing, while the Deceleration one is the least energy-consuming, among all the three kinds of strategies.
Finally, it deserves noting that the data in Table 3 can help us to find the optimal obstacle-avoidance strategy.For example, if the movement time of the robot is limited, we can choose the Acceleration strategy ( 0  2 = 4).If the power supply is not so sufficient, the Deceleration strategy can be selected instead.

Conclusions
The MDP-based probabilistic formal models are constructed for three different kinds of obstacle-avoidance strategies for a mobile robot in an uncertain dynamic environment.The PCTL formulae are then built to perform analysis in the PRISM software, which yields the probability-time curves for the obstacle-avoidance paths.In addition, the time of obstacle-avoidance output by PRISM is taken as input to conduct simulations in MATLAB, which verify the correctness of the results obtained by PRISM.Further, both the minimum time and the energy consumption for the robot to reach its destination under each obstacle-avoidance strategy are obtained by analyses in PRISM.Finally, the time or energy based optimal condition is thereby applied to determine the optimal obstacle-avoidance strategy under diverse circumstances.It deserves noting that the present paper only verifies the probabilistic obstacle-avoidance strategies that involve a mobile robot and a single dynamic obstacle.In practice, there may be more complex obstacle-avoidance conditions.For example, a single mobile robot may encounter several dynamic obstacles, or, even worse, a swarm of mobile robots may have to go across a terrain with multiple dynamic obstacles.These complex cases require more complicated probabilistic formal verifications.The present paper lays a good foundation for these future works.

Figure 1 :
Figure 1: Schematic diagram of a simple MDP.

Figure 6 :
Figure 6: Formal model of the Acceleration strategy.

Figure 7 :
Figure 7: Formal model of the Deceleration strategy.

Figure 8 :
Figure 8: An example of obstacle avoidance.

Figure 9 :
Figure 9: The probability-time curves of the three obstacle-avoidance strategies.

8 Mathematical
Consumption.Here, the minimum time and energy consumption for the robot to go

Figure 11 :
Figure 11: The probability-time curves of the three obstacle-avoidance strategies.

Table 1 :
Simulation results of the acceleration strategy.

Table 2 :
Simulation results of the deceleration strategy.

Table 3 :
Minimum time and energy consumption.
, : Two timers  :It represents the region between  and , where  ∈ {  ,  0 ,  1 ,  2 ,  3 ,  4 ,   } and  ∈ { 0 ,  1 ,  2 ,  3 ,  4 ,   }  : A state index of the dynamic obstacle, which means the obstacle locates at region    : The time for the obstacle to pass region   Safe: The security index.If the value of Safe is 0, it means that an obstacle exists in the Dangerous Region and the robot needs to avoid collisions Conflict: The collision index.If the value of Conflict is 1, it means the robot collides with an obstacle  : It represents the region between  and , where  ∈ {  ,  0 ,  1 ,  2 ,  3 ,  4 ,  5 ,   } and  ∈ { 0 ,  1 ,  2 ,  3 ,  4 ,  5 ,   }  : A state index of the robot, which means the robot locates at region    : A state index of the robot.It means the robot locates at region   with operation errors  : The time for the robot to pass region    : The time for the robot to pass region   with operation errors.