DIMDP: A Driving Intention-Based MDP Service Migration Model under MEC/MSCN Architecture

Service migration is one of the key topics of Internet of Vehicles (IoV). MEC (mobile edge computing), which is carried on the roadside unit (RSU), could serve as a service provider and provide a V2I (vehicle to infrastructure) cooperation service. To solve the high migration rate caused by the vehicle’s high mobility feature, MSCN (mobile secondary computing node) framework is defined. To study service migration features under the MSCN framework further, in this paper, road vehicle’s motion features, e.g., driving intention and regional traffic condition, are introduced to construct the Markov decision model, which is used to explain the service migration decision procedure, while DIMDP (driving intention-based MDP) is defined. Corresponding cost functions are defined, while the optimal object is given. A two-way road scenario is selected as a typical scenario. NS3 platform is employed to fulfill the simulation process. Simulation results show that the proposed service migration strategy performs well and is intensive to vehicle density change.


Introduction
With the rapid growth of vehicles, road safety has become a problem faced by countries all over the world. Vehicle safety applications by means of real-time information exchange between vehicles are techniques for avoiding tra c accidents.
Although cooperative vehicle infrastructure system (CVIS) is considered an e ective approach to support vehicle safety applications, it has to meet the requirements of real-time ability, reliability, and service continuity.
For non-local service requests, the traditional centralized cloud computing network has defects in terms of delay and throughput. erefore, in order to match the needs of realtime driving decision making, scholars have introduced edge computing technologies such as mobile edge computing (MEC) and fog computing (FC) into CVIS. Also, the edge computing server is con gured on the side close to the user, deployed at the end of the base station (BS) and roadside unit (RSU). e back-end service information can be calculated and maintained at the edge node, thus providing relatively adequate computing ability [1] and supporting computational-intensive and time-sensitive operations or exible deployment of applications and services. But for non-local services, MEC needs to maintain a certain service migration rate to reduce service delay. However, the high mobility of vehicle nodes is a negative impact on the e ciency of feedback information and service migration rate. e reason is that the object vehicle may leave the coverage area of the previous MEC, which undertakes computing tasks. To overcome this problem, in our previous work [2], a new concept, mobile secondary computing nodes (MSCNs), was de ned, while a three-layer framework is constructed.
Obviously, the relative speed of MSCN and its surrounding vehicles is lower than that of RSU and its surrounding vehicles and partially solves the service migration problem. However, the corresponding migration mechanism is di erent from that of traditional MEC architecture. Hence, we discussed the service migration problem under MSCN architecture [3] and proposed a MDP (Markov decision process)-based service migration strategy. However, in [3], the service migration strategy was designed according to the relative distance between vehicles and did not consider the vehicle's moving feature and motion model. Obviously, service migration decision making is influenced by the traveling trace of the vehicle, which is the reflection of driving intention and is affected by regional vehicle density. Since the vehicle prefers to go to places with good traffic conditions, we can estimate the vehicle's intention by considering the vehicle's moving feature and motion model.
Furthermore, the MDP model is considered an effective approach to express service migration progress. By establishing a state space that conforms to the vehicle's moving feature and combining the motion model, a trade-off is made between the cost of migration and the benefits after migration to obtain the optimal service migration strategy. erefore, in this paper, driving intention-based MDP service migration strategy under MSCN architecture is proposed. Specifically, our research contributions are as follows: According to vehicle driving intention and regional traffic condition factors over the dynamic feature of service requirement, a MDP-based service migration method, DIMDP (driving intention-based MDP), is proposed. A two-way road scenario, which includes four microscopic events, namely, following, turn left, turn right, and U-turn, is selected as the typical scenario. e rest of the paper is structured as follows. Section 2 presents the related work, and Section 3 discusses the service migration using MDP based on MSCN. e service migration algorithm under MSCN is proposed and applied to traffic for service migration as an example in Section 4. en, experimental simulation and result analysis are presented in Section 5. Finally, the conclusion is given in Section 6.

Related Work
We briefly investigated the existing literature on service migration from the perspective of service delay and trajectory prediction.
Yu et al. [4] prioritized MEC services and proposed a partial dynamic optimization algorithm (PDOA) service migration strategy calculation algorithm, which calculates the migration strategy that minimizes the average delay of long-term service by predicting the mobility of vehicles and achieves good results when the density of vehicles is small. However, the algorithm does not consider the impact of service interruption caused by the long migration time when the vehicle density is high.
Ge et al. [5] proposed the FEE algorithm to calculate the best service migration strategy, which takes into account the current and future time delays in N time slots and the service migration interface between vehicles and changes the focus of the migration strategy by setting up weight coefficients for the delay of each time slot. e algorithm can achieve better results by increasing the number of time slots, which is not obvious when the number of time slots is 5. However, the algorithm does not consider the frequency of service requests, so the algorithm may not achieve better results when services require frequent requests.
Nadembega et al. [6] proposed a mobility-based service migration prediction (MSMP) model, which split user requested service into several portions for service migration by estimating the throughput that the user could receive in advance.
Xu et al. [7] investigated path selection for seamless service migration and proposed a path-selection algorithm to jointly optimize both interests of the network plane and service plane and designed a distance-based filter strategy to eliminate undesired switches in advance to improve the scalability of the proposed algorithm.
Recently, the MDP model is considered an effective approach to express service migration progress. Corresponding researchers considered both user mobility features and MEC-based service requirements to establish decisionmaking strategies [8]. Moreover, cost function definition methods are proposed [9]. Combined with MDP and cost function definition. Taleb et al. [10] proposed a service migration decision algorithm to verify the effectiveness of Follow Me Cloud and provided guidelines for the method in this paper.
However, the existing MDP-based research has not built a model suitable for the CVIS, which considers the vehicle's moving feature and motion model.
In view of traffic simulation, vehicle motion models are categorized into microscopic [11] and macroscopic [12]. Microscopic models focus on the dynamic features of the separate vehicles, while macroscopic models consider traffic features, such as the traffic density, average velocity, flow, and so on. Hence, in this paper, we introduce density factor into the vehicle intention prediction model, thus constructing a microscopic motion model. On the other hand, to establish the MDP model, we borrow from the idea of the cellular model [13], which has adopted the time-space discretion method, to construct a microscopic motion model.

Service Migration under MSCN Framework
According to the percentage of migrated content, service migration can be divided into partial migration and full migration. Different migration methods bring different advantages and disadvantages. Since partial migration involves real-time decisions about which part of the data to migrate, this paper only considers the overall migration of the service.

MSCN Framework.
e MEC-based architecture, MSCN, proposed in our previous work is shown in Figure 1.
Here MSCN framework includes three layers, which are MEC layer, MSCN layer, and general vehicle layer. Experimental verification shows that MSCN can provide reliable RSU-oriented services and significantly improves both communication performance and computing efficiency [2].

MEC Layer.
e MEC layer provides computing and storage resource to road vehicle. Meanwhile, the service provider could provide vehicle-oriented service via the MEC interface.

MSCN Layer.
Selected vehicle nodes, such as city buses, taxis, and so on, are used to construct the MSCN layer, which could provide local computing resources. MSCN nodes collect BSMs sent by surrounding vehicles, then fulfill information fusion, and upload corresponding results to the MEC layer.

General Vehicle Layer.
e general vehicle layer includes all road vehicle nodes, which are equipped with DSRC/LTE-V blocks, and could communicate with each other and RSU directly.

MDP Model Based on Two-Way Lane.
In this paper, we select a two-way lane scenario to discuss the service migration problem. e problem involves the following assumption conditions: Object region is full C-V2X coverage, which means that all on-road vehicles could communicate with RSU. All road vehicles are equipped with a C-V2X module and communicate with RSU via the PC5 interface.
ere are enough public traffic vehicles to serve as MSCNs, which communicate with RSU and other onroad vehicles via C-V2X technologies.
RSUs communicate with each other via wired networks, such as fiber backbone and Ethernet. e relationship between users and service providers is expressed by the distance factor. A service migration event should occur when d(t) > N. In this paper, the maximum value of N is set as 10. MSCN's dwelling time is subject to an exponential distribution with a mean value equal to 1/μ. According to the above assumptions, the two-way lane MDP model is constructed. State space of service migration under the MSCN framework is shown in Figure 2(a), while corresponding state transition progress is denoted in Figure 2(b). As shown in Figure 3, service migration progress under MSCN architecture includes two migration levels, is denoted as social vehicle-oriented migration and public vehicle-oriented migration.
Social vehicle-oriented migration: a social vehicle changes its association MSCN (public vehicles), for example, MSCN1⟶MSCN2 as shown in Figure 3(a). It should be noted that social vehicle can only migrate to the MSCN with the same cruising direction. Public vehicle-oriented migration: a public vehicle (MSCN) changes its association RSU, for example, RSU1⟶RSU2 as shown in Figure 3 As shown in Figure 2

Mobile Information Systems
State transition probability matrix is defined as where p ij , i, j ∈ [0, 4N − 1] is the transition probability of state i and j. In this paper, we take into account the driving intention of the vehicle to obtain the transition probability in the next section. Assume that state probability of state π n is χ n , and State transition progress could be denoted as us, we can obtain the row probability vector χ n of states at the migration strategy N according to equations (3) and (4).

Vehicle Motion Features.
Obviously, service migration progress is decided by vehicle motion features, e.g., cruising direction, cruising speed, and relative speed. For MSCN architecture, two kinds of relative speed, the relative speed between social vehicles (users) and public vehicles (MSCN) and relative speed between public vehicles (MSCN) and RSU, should be considered.
According to [14], for traffic participants, the probability of the specific event in the time interval [t, t + Δt], is defined as where is the number of events that occur per unit time interval under a certain condition and I is the situation-related risk indicator function.
Although equation (5) is constructed for driving risky evaluation, it could be used to express state transition probability. Hence, we define a situation indicator function I s , assuming that traffic prefers to go to places with better traffic conditions and regional traffic condition C traffic , to replace risk indicator function I, as shown as follows: where d I is obtained according to path planning information and C traffic ∈ (0, 1] is real-time traffic information, defined as   Mobile Information Systems 5 where v re(t) (v, v MSCN ) is the relative speed between social vehicles (users) for time t and public vehicles (MSCN), while v re(t) (v MSCN , 0) is the relative speed between public vehicles (MSCN) and RSUs for time t.v limit is the maximum speed limit on road. v is the average cruising speed of vehicles in coverage of corresponding MSCN. v MSCN is the average cruising speed of MSCN in coverage of corresponding RSU. e worse the traffic condition is, the higher the C traffic value should be. Note here that we assume that path change is a result of bad traffic conditions.
In view of service migration, according to vehicle cruising direction, the migration area could be divided into two parts, namely, the forward area and the backward area. As mentioned earlier, here we consider four kinds of events: the following, turn left, turn right, and U-turn. Here we categorize the following, turn left, and turn right as forward area migration inclining events while U-turn as backward area migration inclining events. en, we define vehicle states as π n , n ∈ [0, M − 1], where M is the number of states, and p ij , i, j ∈ [0, M − 1], is the state transition probability, which is denoted as where π i is the state of time t and π j is that of time (t + Δt).
Considering vehicle's inclining events, forward or backward, and with the assumption that traffic prefers to go to places with better traffic conditions and regional traffic conditions C traffic , the state transition probability proportional to d I is defined as follows: where p if and p ib are occurrence probabilities of forward event state π f and forward event state π b . en, p if and p ib could be denoted as Substitute (8) into (11); then, .

Migration and Transmission Cost.
Obviously, the optimization target of the service migration strategy is to minimize migration and transmission costs, and thus it could improve service performance.
Here we assume that the dwelling time of vehicle node i is calculated as where r t,rsu is the coverage radius of vehicle i's home RSU and v vehicle (i) is the cruising speed of vehicle i.

Mobile Information Systems
We define two cost functions, namely, transmission cost function C trans (d) and migration cost function C m (N), as follows: where D trans (d) is the message transmission delay between the user vehicle node and service provider, at a distance d, (f ) is the packet transmission frequency, and N is the migration distance. Migration progress should occur if d > N. D m (N) is the migration delay function, while I m (N) is the service interruption function, defined as follows: where R is the data transfer rate, λ m is the size of the data that needs to be transmitted during the service request process, λ f represents the size of data frame structure waiting to be sent in the queue, and β represents the number of the frame structure, which is affected by network congestion. Message transmission delay is the sum of dissemination delay D t (d) as shown in (16) and queuing delay D q (d), as shown as where d c , which is a constant, represents the wireless communication delay between the vehicle and the RSU. en, we define D trans (d) as Average total cost function, where the state probability of state π n , χ n , is used as a weight to calculate the weighted average of cost, is defined as while the optimal objective is to minimize the average total cost as follows: e pseudocode of optimal strategy is shown in Algorithm 1, while corresponding variables and functions are explained in Table 1. Firstly, the algorithm obtained the state transition probability matrix P according to vehicle driving intention and obtained the row probability vector χ n according to (3) and (4) in different migration distance N. en, the average total cost C a is obtained according to (18). Finally, we compare the total cost at each migration distance to obtain the optimal service migration distance corresponding to the minimum average total cost C a .
Note here that to avoid the impact of the sudden speed change on service migration performance, in this paper, we use regional average speed as the input value of vehicle speed.

Service Migration Strategy.
e proposed MDP-based service migration strategy under the MSCN framework is shown in Algorithm 2, and the corresponding variables and functions are explained in Table 2. First, the current position of each vehicle is obtained and compared with the position of the previous moment to determine whether the MSCN directly under the vehicle has changed. After that, if the MSCN directly under the vehicle changes, determine whether the RSU directly under the vehicle has changed. en, if the RSU directly under the vehicle changes, compare the distance from the current distance with the optimal strategy obtained by Algorithm 1. Finally, once the distance to the optimal policy is reached, service migration is performed and P is updated, with the new service provider as the center. is completes a service migration.
According to the 3GPP suggestion, corresponding parameters are set as e relationship between migration distance N and migration cost with different p values is shown in Figure 4. Here regional average speed is set as 20 m/s. As shown in Figure 4, total cost increases with increasing p, and thus we verify the validity of the cost function. Moreover, the optimal object S, which is defined by (19), varies from 3 to 4. e relationship between migration distance N and migration cost with different regional average speed values is shown in Figure 5.
As shown in Figure 5, the average total cost increases with increasing regional average speed, which expresses the influence of the vehicle's cruising intention. e value range of optimal object S is [2,5] and consistent with the result of Figure 5.

Intersection Control under the MSCN Framework.
Intersection control in the MSCN framework leverages RSU collaboration across multiple intersections and decides Mobile Information Systems 7 Input:  Service request frequency C min e minimal average total cost C a S e optimal migration strategy, a service migration event occurring when d(t) ≥ S get_Pmatrix (pif, N) Obtain the state transition probability matrix P according to pif at migration distance N get_state (P, N) Obtain the row probability vector χ n of states at the migration distance N according to (3) and (-4) Get_cost (Cset, χ n , N) Obtain the average total cost C a (N) according to (18). Cset is λ m , θ m , λ f , r t,rsu , v vehicle , f . Center coordinates of middle cell t1 Last moment t2 Current moment d Distance between service requester and service provider Cset A set of λ m , θ m , λ f , r t,rsu , v vehicle , f pif As explained in Table 1 Vtarget e vehicle member currently traversed get_position (V, t) Get the position of vehicle V at time t get_cellnum ((xt, yt), (xcn, ycn)) Obtain the cell number of coordinate (xt,yt) by taking coordinate (xcn, ycn) as a reference point get_MSCN (cellnum) Obtain the MSCN id corresponding to cell cellnum get_rsu (MSCNid) Obtain the RSU id to which MSCNid belongs mscnMigration A map < vehicle_id,flag > which represents the vehicle who's id is vehicle_id will perform or not service migration at the MSCN level while the flag is true or false rsuMigration A map < vehicle_id,flag > which represents the vehicle who's id is vehicle_id will perform or not service migration at the RSU level while the flag is true or false setMigration e service migration set {rsuMigration, mscnMigration} get_strategy (pif, cset) Obtain the optimal strategy according to Algorithm 1 with variable pif and variable Cset  t1); (x t2 ,y t2 ) ← get_position (V target ,t2); cellnum1 ← getcell_num ((x t1 , y t1 ),(x cn , y cn )); cellnum2 ← getcell_num ((x t2 , y t2 ),(x cn , y cn ));  e intersection control application in the MSCN framework is shown in Figure 6, in which blue vehicles represent upcoming right-turn vehicles after passing through the intersection, red represents upcoming straighttraffic vehicles, and black represents upcoming left-turn vehicles. RSUs deploying MEC services are set up between upstream and downstream intersections, and the distance between RSUs is determined according to their transmission range. e specific data flow is as follows: (1) e road vehicles decide whether to perform service migration according to the DIMDP in the MSCN framework.  Table 3. (4) e MSCN decides whether to perform application service migration based on the DIMDP optimization policy.
(5) e MSCN sends Table 3 to the optimal RSU. (6) Optimal RSU assembles and organizes Table 3 from each cluster and MSCN to obtain Table 4. (7) e adjacent RSUs exchange Table 4 to generate  updating Table 4. (8) e optimal RSU sends updating Table 4 to the traffic light controller and executes intersection control. Table 3 shows the preliminary results after MSCN collects all the normal vehicle nodes and its own application information in its region and conducts statistical sorting. en, the MSCN sends Table 3 to the optimal RSU, which receives Table 3 from each cluster and each MSCN within the cluster, processes, and counts it to form Table 4.

Simulation Setup.
Simulation is done based on the NS-3.28 platform. Simulation parameters are listed in Table 5. We excerpted a 600-meter two-way road used in the simulation as a diagram as shown in Figure 7.
Corresponding communication parameters are listed in Table 6.
Note here that V2X information is one kind of timeliness information. Hence, re-transmission procedure is disabled. Moreover, until today, most C-V2X-based modules do not support power adjusting function and use fixed transmit power, valued as 23 dBm. ree parameters, average backhaul delay (ABL), packet delivery rate (PDR), and effective feedback ratio (EFR), are used to evaluate the performance of the proposed mechanism.
Here effective feedback ratio is defined as the ratio of received effective service message number to uploaded BSM number, while the average backhaul delay is defined as the service response delay, which is the interval between the source node's BSM sending time t bsm and destination node's service response time t service . Considering the influence of BSM dissemination delay, here we employ the average BSM dissemination delay to calculate ABL t ABL as follows: where N is the BSM packet number.
To verify the performance of DIMDP under MSCN architecture, three others are selected as a contrast.
(i) MDP service migration strategy without driving intention under MSCN architecture. (ii) Always migration strategy under MSCN architecture, whose migration condition is set as d (t) ≥ 0.    Figure 7: e diagram of the two-way road used in the simulation. As shown in Figure 8, the p value has no significant effect on packet delivery rate and backhaul delay. e proposed DIMDP method performs better backhaul delay, which means the V2X security applications can respond faster.
As shown in Figure 9, with the increase in road vehicle density, both the ABL and PDR of the proposed DIMDP method under MSCN architecture are nearly equal to a constant and perform better, which means it can still maintain excellent and stable performance in scenes with high vehicle density.
As shown in Figure 10, the higher the vehicle velocity is, the lower the effective feedback ratio should be. Moreover, the effective feedback ratio of the proposed DIMDP method under MSCN architecture is higher than that of other methods.

Conclusion
V2X safety applications have time-sensitive characteristics, and their application performance largely depends on the effectiveness of the information sharing issues. Due to the high mobility of vehicle nodes, service providers, such as MEC servers and cloud servers, could not disseminate the useful messages to vehicle nodes on time. Fortunately, a reasonable calculation framework and effective transmission guarantee measure are useful to solve the above problems. MSCN framework, which uses public vehicles as mobile service providers, could decrease the service migration rate and maintain service continuity. On the other hand, although the MDP model could be used to express service migration progress, the vehicle motion features should be considered in the construction of the MDP model.
In this paper, we use vehicle driving intention and regional traffic conditions to express the vehicle's mobile tendency. A complex cost function, which considers both transmission factor and migration cost, is defined. Simulation results show that the proposed DIMDP model performs well.
In the future, we shall work on the vehicle motion model, thus further refining our model and improving its performance.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper. Mobile Information Systems 13