Receding Horizon Control with Extended Solution for UAV Path Planning

The receding horizon control (RHC) greatly reduces the planning time and achieves great success in UAV online path planning because of rolling window optimization. However, due to its small range of path search in the time window, UAVs cannot cope with environments with uncertain obstacles and multiple flight constraints. Therefore, the receding horizon control with extended solution (RHC-eS) method is proposed for UAV path planning based on the traditional RHC. This method first designs the path expansion mechanism, which not only expands the search range of feasible solutions but also ensures the real-time performance by the two-way search strategy. Secondly, in order to increase the richness of solutions, the crossover and directional variation strategy of Genetic Algorithm (GA) are integrated. Finally, the Sequential Quadratic Programming (SQP) method is used to optimize the objective function. The simulation results of UAV path planning in simple and complex environments certify that the proposed method can obtain shorter, safer, and smoother paths compared with the existing methods.


Introduction
With the advent of the era of intelligence, information, and automation, unmanned aerial vehicle systems (UAVs) play an important role in military, power, environmental assessment, and other fields. Especially in modern military, UAV is used for remote detection and target shooting, which realizes high combat value with low cost. Path planning is one of the key technologies of UAV autonomous flight research [1], and its performance directly determines whether the task can be completed accurately and safely. Therefore, the research on UAV path planning has high theoretical significance and practical value. UAV path planning refers to searching a smooth path that satisfied the requirements in the flight environment [2,3]. At present, related scholars are endeavored to finding more effective and feasible path planning methods, which can be divided into two types: geometric methods and heuristic methods [4]. For example, Voronoi diagram and Dubins curves are the typical geometric methods.
Heuristic methods include A * [5,6], D * [7], Rapid Random Tree (RRT) [8], Genetic Algorithm (GA) [9], Ant Colony Optimization (ACO) [10][11][12][13], Particle Swarm Optimization (PSO) [14,15], and other methods [16][17][18][19]. However, these algorithms have some shortcomings that make them not well applicable to solve path planning problems in online and dynamic environments. For example, Shin et al. [20]. used the improved PSO for online and local path planning, but this method did not perform in global planning. In addition, Ma et al. [21] proposed an improved GA for path planning, but it can only be used in static environments where the environment is completely known and cannot effectively cope with real-time environment. In recent years, the receding horizon control (RHC) [22] is widely used in UAV online path planning. Its basic idea is that as the time window rolls forward, the local optimization is carried out in the rolling time window, by using the real-time updated environmental information and the path feedback information of the previous window until reaching the destination [23]. Major advantages of RHC for path planning include better real-time planning ability, fast algorithm response speed, and effective path information feedback ability. However, it cannot adapt to complex and unknown planning environment, and the real-time performance will be reduced in complex situations. For example, Bryce et al. [24] consider receding horizon control for online path planning, but this method can only be applied in the dynamic environment where the map is completely known, and it cannot be applied to the environment with unknown threat sources. Zhang et al. [25]. presents an adaptive receding horizon control method for path planning. Although it improves the efficiency of planning in static environment, it cannot guarantee the real-time performance in dynamic, unknown and complex environment. Kristoffer et al. [26] introduced the improved receding horizon control for local path planning. The path generation speed of this method is fast, but it has the problem of small search range for feasible paths, which is difficult to effectively adjust the path and avoid obstacles in complex environments. The above several path planning methods based on the RHC can be used for online path planning effectively. However, in the face of unexpected situations (such as sudden birds in the air and enemy raids,) and uncertain environment with many obstacles (such as forest protection and navigation), it is unable to avoid obstacles effectively, and the real-time performance will be reduced. This is mainly due to the search scope of the path is small, and the path generated is single and not feasible. In view of the above problems, this article proposes a receding horizon control with extended solution (RHC-eS) method for UAV path planning. The process of the method is shown in Figure 1. Firstly, the path expansion mechanism is designed to solve the problem of reducing the real-time performance of RHC method in complex environment. Next, crossover and variation strategies are integrated to solve the problem that it is difficult for the RHC method to find a feasible solution in the dynamic uncertain environment. Finally, through extensive experiments, the results revealed that RHC-eS has faster and safer performance in planning shorter smooth paths than D * and RHC.
The main contributions of this paper can be concluded as follows: (1) path expansion mechanism is designed. By expanding the search range of the algorithm and designing a two-way search strategy, the real-time requirements of the RHC method for path planning in dynamic and unknown environments are guaranteed. (2) The crossover and variation strategies of Genetic Algorithm are integrated into the RHC method. By increasing the richness and diversity of paths, the probability of finding feasible solutions in the path planning of the RHC method is improved. (3) RHC-eS algorithm is proposed. It can flexibly deal with the changeable environment because of its diverse paths and the characteristics of large-scale search for paths. Experiments in static, dynamic, and unknown environments verify the correctness and effectiveness of the proposed method. Other chapters are arranged as follows: in Section 2, the relevant knowledge of UAV path planning is introduced. In Section 3, the receding horizon control with extended solution method is presented in detail. Section 4 gives a series of experiments to validate the techniques in static, dynamic, and unknown environments. Section 5 concludes the paper.

Relevant Knowledge
In this chapter, the UAV path planning problem and the method based on receding horizon control will be briefly introduced.
2.1. Path Planning Problem. The UAV path planning problem is generally as an optimization control problem, which consists of an objective function, a set of boundary conditions, and control constraints [27].
In the environment with unknown and high-speed motion threats, there are high requirements for real-time and obstacle avoidance of UAV. The path length is usually as the optimization objective function, as listed below.
where ðx, yÞ is the path coordinate point, in particular, ðx 0 , y 0 Þ is the position coordinate of the UAV, and ðx f , y f Þ is the end point coordinate.
Considering flight safety and machine performance conditions, the following constraints shall be satisfied in UAV path planning.
(1) Obstacle avoidance constraint The obstacle is expanded into a sphere. In order to avoid collision between UAV and dynamic obstacle, the distance between them is defined as where L uav is the wingspan length of the UAV, r d and ðx d , y d Þ represent the radius and position coordinates of the dynamic obstacle, respectively. There are similar calculation methods for obstacle avoidance of static obstacles and unknown threats.
(2) Velocity constraint UAV must navigate within its performance, so the velocity satisfies the following conditions. Under the condition of ensuring that the UAV meets the flight speed constraints, it pays more attention to obstacle avoidance and finding an optimal path. Therefore, the acceleration is no longer accu- (3) Flight dynamics constraint where r flight represents the curvature radius of flight and r uav represents the minimum turning radius of the UAV.  Figure 2. The steps are as follows.
Step 1. At sampling time t, initialize UAV and environment information, construct rolling time window T w , and set time step T n , where T w ≥ T n Step 2. In the time window, the initial reference path f ðx, yÞ is randomly generated towards the target direction Step 3. Establishing optimization objective function min Fðx, yÞ with path length Step 4. The objective function is solved by the optimization algorithm (Sequential Quadratic Programming method in this paper), and the optimal objective function F * ðx, yÞ and optimal path f * ðx, yÞ are obtained Step 5. The optimal path is divided into two parts: the first part is the flight path f flight in ½0, T n time. The second part is the prediction path f predict in ½T n , T w time. Execute flight path f flight and scroll the window forward in time steps Step 6. Update UAV location and environment information Step 7. Enter the next sampling time. Judge whether to reach the end point. If not, take the predicted path f predict as the feedback correction of the initial feasible path and return to step 2. Otherwise, the algorithm ends and outputs all flight paths f flight In step 2 of the RHC method, the reference path is randomly generated along a single target direction. In this way, it is difficult to find a feasible path in the face of dynamic and unknown threat environment, and it will reduce the optimization speed of the algorithm and cannot guarantee the real-time performance. Therefore, we propose a new method: receding horizon control with extended solution (RHC-eS). This method has the following similarities and differences with the receding horizon control.
(1) The path expansion mechanism is designed to expand the search area of the path (2) The crossover and variation are integrated to increase the richness and diversity of search path. This method is more suitable for various dynamic uncertain flight environments

Receding Horizon Control with Extended Solution
We design the path expansion mechanism and integrate the crossover and variation strategies to solve the problems that the receding horizon control method is difficult to find feasible solutions and reduce real-time performance in unknown environment.

Path Expansion Mechanism.
The reference path generated by the RHC is the initial prediction of the path. The path expansion mechanism is used to generate reference paths and increase their number. Its definition is shown in Equation (5), and the diagram is shown in Figure 3. where , θ is the angle between the target direction and the flight direction, and β is the angle between the target direction and the X axis direction. When the flight direction is between the target direction and the positive direction of the X axis, the angle is β − θ, and when the flight direction is between the target direction and the positive direction of the Y axis, the angle is β + θ. L max is the maximum flight distance of UAV in one second, and T w is the rolling time window. The two-way search strategy of path expansion mechanism is to change the path search direction by controlling the size of angle θ. When the surrounding environment is less threatened, the path is searched only in the target direction (θ = 0°) to improve the search speed of UAV. When there are unknown obstacles in the environment, multidirection search (θ ≠ 0°) is used to enhance the search ability of feasible solutions.
In path planning, most algorithms often use random methods to generate initial prediction paths. On the contrary, the proposed method in this paper makes the path generate in a certain angle range toward the destination, which greatly improves the probability of finding the optimal solution.  Figure 1: Receding horizon control with extended solution for UAV path planning.

International Journal of Aerospace Engineering
However, in order to cope with the complex environment, we integrate the crossover and variation strategies to improve the richness and diversity of paths.

Crossing and Directional
Variation. Crossover and variation are the essence of Genetic Algorithm which imitates the evolution mechanism of nature [28].
The cross strategy can improve the global search capability for feasible solutions. Taking the reference path f ð x, yÞ as the initial population, we mixed two crossover methods: average crossover [29,30] and single point crossover to enrich the population.
Firstly, the average crossover formula for f ð x, yÞ is given as where ð f ð x, yÞÞ m represents the m optimized reference path. The number of reference paths is consistent with the number of search directions, that is, one reference path is generated in each search direction. Then, the single point crossover formula for f ð x, yÞ and f ac ðx, yÞ is Equation (7). The idea of single-point crossover is to select two paths according to the crossover probability pc and then randomly select the position points (excluding the beginning and end points), split the path, and exchange the right part to obtain two new paths.
Note that the operator f•g in Equation (7) is used to calculate the single point crossover between two paths. Finally, after integrating pathsf ð x, yÞ, f ac ð x, yÞ, andf sc ð x, yÞ, variation operation is carried out. Variation can increase the diversity of solutions [31]. We no longer use the traditional method to randomly generate new path points in the variation part, but develop directional variation strategy combined with the path expansion mechanism. A comparison of the two strategies is shown in Figure 4. The yellow triangle represents the UAV, and the red pentagon represents the end destination. It can be seen from the figure that the direction of directional variation is towards the destination, so the results tend to be better. However, random variation has great uncertainty, such as random direction 1, which cannot guarantee the quality of variation results. The steps of directional variation are as follows: Step 1. The variation probability pm is used to determine whether to carry out variation. If feasible, enter Step 2; otherwise, the path will not undergo variation operation.
Step 2. The variation path segment is randomly selected. Then, randomly select a variation direction from the two directions corresponding to θ = 30°and the target direction corresponding to θ = 0°.
In the variant segment, the path expansion mechanism is used to regenerate the reference path in the selected direction, so as to obtain a new variated path f dv ðx, yÞ.
Step 4. Integrating paths together maximize the population size to obtain the total predicted path group f z ðx, yÞ = ½ f ð x, yÞ, f ac ðx, yÞ, f sc ðx, yÞ, f dv ðx, yÞ. In order to obtain the smooth path, we use the quadratic Bezier curve to smooth the total predicted path group f z ðx, yÞ with reference to literature [24]. The smooth prediction path is defined as follows.
where k is the positional parameter and ðx 0 , y 0 Þ, ðx 1 , y 1 Þ, andðx 2 , y 2 Þrepresent the coordinates of the first, middle, and end control points. The path point in f z ðx, yÞ is used as the control point of quadratic Bezier curve. Each path consists of multiple quadratic Bezier curves. Therefore, the total group of smooth prediction paths can be described as where n represents the number of quadratic Bezier curves.  In order to meet the flight dynamics constraints of UAV, the following constraint is defined.    where r bez is the radius of curvature, _ xðkÞ, _ yðkÞ, € xðkÞ, and€ yðkÞ represent the first and second derivatives of formula f z ðxðkÞ, yðkÞÞ to x and y, respectively.
The effects of crossover and variation on the path will be described in detail in the experiment section.

Receding Horizon Control with Extended Solution.
The receding horizon control with extended solution takes the path length as the objective function. The objective function and constraint conditions are expressed as s:t: where k f sz ðx, yÞk represents the total length of the smooth predicted path andL f is the linear distance from the last predicted path point to the target point.
Since the smooth predicted path is composed of curves, it is difficult to calculate the length accurately, so the broken line length is used to approximate the length of the curve. The smooth predicted path length is estimated as follows.
Note that when i = w, ðx w , y w Þ represents the end point of the smooth predicted path.
The distance from the final destination of UAV to the final path point is given by According to reference [24], the constrained objective function Fðx, yÞ is solved by the Sequential Quadratic Programming (SQP) method. Putting together all of the elements discussed in this section, the RHC-eS works as follows. To begin, initialize the UAV and environment information. In addition, establish the rolling time window and set the time step size. The algorithm then loops through several actions. Firstly, according to the UAV position, the path expansion mechanism (5) is used to generate the reference path. Next, a mixed crossover is performed using (6)-(7), and the directed variation is performed as described in Section 3.2. Then, the Sequential Quadratic Programming method is used to solve the objective function to obtain the optimal prediction path. Keep the predicted path within the time step; after that, scroll the time window forward, and update the UAV and environment information. This process is repeated until near the destination. When the algorithm meets the termination condition of the cycle, the distance between the UAV and the destination is within a time step. By replacing the last path point with the destination coordinate, the UAV can fly directly to the destination.
In this way, the situation of UAV flying over the destination is avoided. Finally, when the UAV reaches the destination, the algorithm is completed. The steps of RHC-eS for UAV path planning are given in Algorithm 1.

Experimental Environment Settings.
In the twodimensional map 100 × 100, static obstacles are randomly distributed in the map (except the start and end points), with a radius of 1 to 5 meters. The initial position, size, speed, and number of dynamic obstacles are specified. The initial position of the UAV is ðx 0 , y 0 Þ = ð0, 0Þ, and the destination is ðx f , y f Þ = ð100, 100Þ. The speed range of UAV is v max = 15m/s, the minimum turning radius of UAV is taken asr uav = 5m, and wingspan is L uav = 1m. For the best algorithm performance, the values of rolling time window T w , time step T n , crossover probability pc, and variation probability pm are set to 3 s, 1 s, 0.1, and 0.5, respectively. Position parameter of Bezier curve is taken as k = 0:1, 0:2, ⋯, 1. The number of reference paths m is 7, and multidirectional path search angle is taken as θ = 0°, 30°, 60°, 90°.
We set up six obstacle environments for experiments, three simple environments, and three complex environments. Simple environment includes dynamic obstacles and static obstacles. The parameters of dynamic obstacles are shown in Table 1. The location and size of static obstacles are generated randomly, and the number is set to 1.
In order to compare with the simple environment, the complex environment chooses to add unknown dynamic obstacles in the simple environment, as shown in Table 2.

Experimental
Results and Analysis. Experiments are carried out in simple and complex map environments to verify the feasibility and effectiveness of the proposed method, and the results are compared with the receding horizon control method and D * method. In order to eliminate the influence of random and other contingency factors on the algorithm,   Tables 3 and 4, where "-" means that the algorithm does not find a feasible path, and the statistical results cannot be obtained. The experimental comparison results of RHC, RHC-eS, and D * in simple environments are shown in Table 3.
Combined with Figures 5-7 and Table 3, the following conclusions are drawn.

Data Analysis of Path Length
(i) In a simple environment, the three methods can plan a safe path. Overall, the path planning length of RHC-eS is the shortest, followed by RHC (ii) In case 1 and case 2, the path of D * is worse than that of the other two methods. This is because the path planning method of D * lacks forecasts of potential threats in the next period of time, and passive avoidance in replanning, resulting in a longer path. However, the other two methods adopt realtime and rolling window planning; both of which can predict the danger of the coming period and can replan the path in advance, so the path is better (iii) In case 3, the dynamic obstacle poses less threat to the path, and three algorithms can quickly find the optimal path

Data Analysis of Algorithm Running Time
(i) On the whole, RHC-eS takes the shortest time and is much shorter than RHC. The main influencing factor is the path expansion mechanism. When there is no obstacle threat, the mechanism directly uses    Table 4.
From Figures 8-10 and Table 4, results can be drawn as follows.
(1) Data Analysis of Path Length. In unknown environments, RHC-eS can still find a feasible path when no feasible solution is found in RHC and D * . This is due to the integra-tion of crossover and variation strategies. In the environment with unknown obstacles, genetic strategy increases the diversity and globality of path search, making it easier for the algorithm to find feasible solutions. However, the search range of the RHC is, small and the search direction is limited. The D * is disturbed by unknown dynamic obstacles in the replanning and fails to avoid obstacles for many times. Therefore, neither of them can cope with the complex environment.  (2) Data Analysis of Algorithm Running Time. RHC-eS is superior to the other two methods in the overall operation time. In environment case 5, the operation time of RHC-eS is longer than that of RHC. However, when the feasible path is not found in the D * , the planned path of RHC-eS is much better than that of RHC. When RHC and D * fails to find a feasible path in case 6, RHC-eS can still find the optimal path quickly.
(3) Overall Analysis Results of Unknown Environment Experiments. The results confirm the effectiveness of RHC-eS in real-time planning with unknown dynamic obstacles, and it is better than D * and RHC method.

Parameter Sensitivity.
In order to test the influence of rolling time window T w , crossover operator pc, and variation operator pm on algorithm performance. In the case of other   Figure 11.
It can be seen from Figure 11 that in both environments, the path length has a minimum value near the size of the rolling time window of 3. Although the path length is shorter at the size of 7, the time spent significantly increases, which cannot guarantee the real-time requirements. Therefore, given the influence of time and path length, the size of rolling time window is selected as 3.
In the environment case 2 and case 5, the effect of cross operator pc on path length and time is shown in Figure 12.
From Figure 12, the path length oscillates with the increase of pc value, and the time presents a floating upward trend in two environments. Since the mixed crossover mode has a great influence on the path planning effect, when the

12
International Journal of Aerospace Engineering crossover probability increases, it is easy to cause the oscillation of the path length. Therefore, the test results of the two environments are combined to determine the crossover probability of 0.1. Under environment case 2 and case 5, the influence of variation operator pm on path length and time is shown in Figure 13. It can be seen from Figure 13 that in both environments, time increases with the increase of pm. The planning path length decreases rapidly when 0.4~0.7 is taken in case 2. In case 5, the path length tends to be stable after 0.5. When the variation probability is large, the reason for the better planning result is that we adopt the directional variation method, which increases the chance of variation towards a good solution and reduces the uncertain impact of variation. Based on the above analysis, the variation probability set to 0.5.
In summary, we takes T w = 3, pc = 0:1, and pm = 0:5 to ensure the better performance of path planning.

Crossover and Variation
Test. In this paper, experiments are carried out in a variety of environments to illustrate the impact of crossover and variation on path planning. In order to more clearly show the changes of the path, a group of experiments in which crossover and variation have a great impact on the path are selected for explanation. The environment settings are shown in Table 5, including two known dynamic obstacles.
Other parameters remain unchanged, and the crossover probability and variation probability are changed for testing. The results are shown in the table below.
The following results can be obtained from Figure 14 and Table 6.
In terms of time, the path planning time of single crossover experiment Figure 14(a) and single variation experiment Figure 14(b) is about twice that of fusion crossover and variation experiment Figure 14(f). Comparing the path length, although the time is shortened when changing the crossover and variation probability, the planned path becomes longer (Figure 14(d)), and even the path is not found (Figure 14(c)). It can be seen from the figure that the fusion crossover and     14 International Journal of Aerospace Engineering variation experiment changed the original infeasible path (Figure 14(e)) and made the UAV find the feasible path again.
To sum up, a single crossover and variation can find a feasible path and change the path, but on the whole, the path planning effect of adding crossover and variation at the same time is better. In conclusion, crossover and mutation are proved to be effective for path optimization.

Conclusion
In order to deal with the path planning of UAV in dynamic uncertain environment, a receding horizon control with extended solution is designed. In this method, the path expansion mechanism is proved to be more effective than one-way optimization. Compared with the traditional receding horizon control, the fusion of crossover and directed variation unlocks the critical bottleneck that is difficult to adapt to complex environment. Our method is verified to be able to avoid threats and obstacles more effectively under uncertainties than the traditional methods. Future work will focus on extending this method to multimachine and threedimensional path planning.

Data Availability
The data used to support the findings of this study can be found at https://github.com/byuflowlab/uav-path-optimization.

Conflicts of Interest
The authors declare that they have no conflicts of interest.