Optimization of Energy Recovery Efficiency for Parallel Hydraulic Hybrid Power Systems Based on Dynamic Programming

In this paper, an optimization algorithm of energy recovery efficiency is proposed for parallel hydraulic hybrid systems (PHHS) using dynamic programming (DP). Global optimal solution of pump displacement and transmission ratio under the known urban drive cycles is obtained by using the DP approach, where the total amount of energy recovery is defined as the cost function, and the pump displacement and the transmission ratio of the torque coupler are defined as the deciding variables. Two major steps are involved in verifying the proposed approach. Firstly, a PHHS Simulink model is accurately obtained by repeated comparison with the bench test. Subsequently, we derive a parallel hydraulic hybrid vehicle (PHHV) from adding a hydraulic hybrid system to an electric vehicle in ADVISOR (advanced vehicle simulator).This vehicle is used to validate the effectiveness of the proposed method in energy recovery efficiency.


Introduction
It has been widely recognized that the fast development of hybrid vehicle (HV) presents a favorable solution to global warming and fossil fuel shortages.As compared to conventional cars with an internal combustion engine, a car with hybrid power train has greater potential to improve efficiencies in fuel economy and reduce emissions from vehicles, since the brake-regenerated energy can be stored and reused by extra power systems [1].Hydraulic hybrid systems are extensively applied in the stop-and-go vehicles such as shuttle bus and refuse trucks [2,3] for their advantages including high power density, high energy conversion efficiency, and high rates of charging and discharging compared with the hybrid electric vehicles which use batteries to store energy.Studies have indicated that PHHS is the simplest way to implement existing nonhybrid architectures as compared to the series and power-split configurations, especially for the commercial vehicles [4][5][6].
Optimal power management (OPM) design plays an essential role in improving hybrid power train efficiency.The current advanced optimal control methods have been developed for OPM, such as those reliant on dynamic programming (DP) [7,8], conventional model predictive control (MPC) [9], quadratic programming and genetic algorithm (GA) [10], and rule-based control [11], all focusing on an integrated methodology of propulsion and braking to improve the fuel economy in a typical urban cycle.It is well known that the vehicles are frequently used to start and stop under the driving condition of the city, and as much as 30%∼50% of kinetic energy is lost during the braking process [12].This provides a strong motivation to develop a more effective energy recovery system for PHHS.The major factors affecting the energy recovery efficiency of PHHS include driving mode, running conditions, torque coupler transmission ratio, and the main hydraulic components parameters [13].Plenty of scholars use hydraulic simulation software to build simulation model of PHHS in order to study the influencing factors [14,15] for energy recovery efficiency of PHHS.Moreover, the optimization strategy for the parameters of the PHHS is matched and analyzed in [16,17].Plenty of studies [18][19][20][21][22] have proved that dynamic programming offers an effective method to improve fuel economy and reduce emissions from hybrid vehicles by properly allocating power between the engine and the secondary power.
The global optimization of the hydraulic hybrid system and the regenerative braking strategy of the vehicle are two separate parts.This paper place its focus on optimizing the parameters of the hydraulic hybrid system to enhance energy recovery efficiency, regardless of the braking energy recovery control strategy adopted in the electric or hybrid vehicle [23,24].The energy recovery efficiency can be improved, after the hydraulic hybrid system is improved based on the dynamic programming algorithm, which is also reflected in the fourth part of the vehicle simulation.
In this paper, a DP-based optimization of energy recovery efficiency algorithm is presented for PHHS.Global optimal solution of pump displacement and transmission ratio under the known urban drive cycles is obtained by DP method, where the aggregate amount of energy recovery is defined as the cost function, and the pump displacement and the transmission ratio of the torque coupler are defined as the decision variables.In order to improve the effectiveness of the vehicle simulation results, simulation tools are required to exhibit the advantages of reliability, robustness, and numerical stability [25].Therefore, we chose ADVISOR established in the Matlab/Simulink environment.Besides, we need to determine the representative driving cycles, considering the various road conditions such as urban area and highway [26], for which we chose the 1015 and UDDS as the driving cycles we used in this paper.Two major steps are included to validate the proposed approach as follows.Firstly, a PHHS Matlab/Simulink model is accurately obtained by repeated comparison with the PHHS bench test.Subsequently, we integrated PHHS into the vehicle model of ADVISOR and got a parallel hydraulic hybrid car.Through simulation verification, we demonstrated that our proposed DPHS based on DP algorithm is effectively improving the efficiency of braking energy recovery.
The remainder of this paper is structured as follows.In Section 2, the PHHS model is presented.The proposed DPbased optimization of energy recovery efficiency algorithm is introduced in Section 3. Section 4 presents the bench test experiment to verity the PHHS simulation model before an analysis is performed on the simulation results of the proposed approach based on the whole vehicle model from ADVISOR.Finally, the conclusions are revealed in Section 5.

Parallel Hydraulic Hybrid Systems (PHHS)
The general configuration of the PHHS is indicated in Figure 1, which consists of an electromagnetic digital valve, a variable hydraulic pump/motor, a high-pressure accumulator, a relief valve, a torque coupler, an electromagnetic clutch, and a tank.The PHHS is connected by the torque coupler with the drive system of the car, and the energy recovery is achieved on the premise of maintaining the original layout of the car chassis.In the entire system, the hydraulic pump/motor is used for the conversion between the hydraulic potential energy and the kinetic energy of the car.When the driver steps on the brake pedal, the controller collects the brake signal, the electromagnetic clutch between the energy recovery system and the vehicle power system is switched off, and the PHHS starts to work.In the meantime, the hydraulic pump/motor is driven by the kinetic energy of the vehicle, and the energy is restored in the accumulator.The braking torque produced by the pump/motor can decelerate the vehicle to a complete stop and the frictional brake would take effect only at a time when the pump/motor is incapable to provide sufficient braking force.When the vehicle starts, the electromagnetic clutch is connected, and the PHHS works.Meanwhile, the energy stored in the accumulator is utilized to drive the hydraulic pump/motor.At this point, the secondary element operates as a hydraulic motor and provides torque for start and acceleration of the vehicle.
The kinetic energy of the vehicle before braking is selected as the main objective of the energy recovery of the PHHS, which is simplified as where  represents the mass of the vehicle and u is defined as the velocity of the vehicle before braking.The hydraulic pump/motor is the major component for conversion between the hydraulic potential energy of the accumulator and the kinetic energy of the vehicle.The torque provided by the pump/motor is calculated as where  / denotes the torque of the hydraulic pump/motor when the vehicle is in either the braking or driving conditions; Δ is referred to as the working pressure of the hydraulic pump/motor;  / is defined as the mechanical efficiency of the hydraulic pump/motor; and  / stands for the displacement of the hydraulic pump/motor. 0 is expressed as the speed ratio of the transmission, and   is the speed ration of the final drive.Air bag accumulator is selected as the energy storage element of PHHS.Based on the Boyle's law, the relationship between the volume and pressure of the hydraulic accumulator is described as where  0 is referred to as the initial pressure of the accumulator;  1 denotes the minimum working pressure of the system;  2 represents the maximum level of pressure in the system;  0 is the volume of the accumulator gas corresponding to the initial pressure;  1 is defined as volume of the accumulator gas corresponding to the minimum working pressure;  2 symbolizes the volume of the accumulator gas correspondent to the maximum pressure;  is expressed as air polytropic exponent value.An assumption is made that the fluid in the hydraulic system is incompressible and the accumulator is working in an adiabatic state; air polytropic exponent value is thus selected as 1.4.
In the process of brake energy regeneration, the amount of energy stored in the accumulator could be obtained by applying the equation as follows: where   is defined as the energy recovery of the accumulator and  represents the ultimate pressure of the accumulator.
The maximum energy stored in the accumulator is realized when  reaches the maximum value.
The energy recovery efficiency of PHHS can be expressed as follows.
The torque coupler is responsible for the power conversion between the hydraulic system and the automobile transmission system.The detained relationship is illustrated as follows: where  1 is referred to as the input torque of the coupler, which is produced by the hydraulic system;  denotes the output torque of the coupler, which is the torque applied to the transmission system of PHHS;   is defined as the speed of pump/motor and  indicates the speed of transmission system; and i stands for the gear ratio of the coupler.
In a parallel hydraulic hybrid vehicle, the recovery of the vehicle's drive and braking energy is accomplished by a hydraulic pump/motor, for which the hydraulic pump/motor functions as both a drive component and a brake component.Combined with the operating characteristics of the parallel hydraulic hybrid system, the hydraulic pump/motor is required to satisfy a number of requirements as listed below: (1) In the initial acceleration phase, the hydraulic hybrid system is able to independently drive the vehicle.
(2) In the brake deceleration phase, the hydraulic hybrid system can improve the regenerative braking capability to the maximum level.
When the hydraulic system drives the vehicle, the displacement of the secondary components is supposed to satisfy the following inequalities: where F is defined as the driving resistance of the vehicle, G represents the acceleration of gravity, f is referred to as the rolling resistance coefficient which is a constant,   is the air resistance coefficient, and  stands for the frontal area of the car. indicates the road gradient. denotes the rotation mass conversion factor of vehicle.
In order to restore the braking energy to the maximum level under urban conditions, it is required to ensure that its braking torque is fully provided by the hydraulic braking energy recovery system during light braking.Therefore, when the braking strength is 0.1, braking strength is defined as the ratio of brake deceleration to gravitational acceleration, and the braking torque provided by the hydraulic pump at the minimum working pressure can meet the braking requirements of the vehicle: where z is braking strength.

Dynamic Programming Algorithm
Dealing with dynamic programming is a multistage decision issue.It will be used to improve the energy recovery efficiency for PHHS.The proposed optimal energy recovery strategy based on DP mainly includes cost function, decision variables, and constraint conditions.Considering the following discrete-time system  ( + 1) =  ( () ,  ()) ,  = 0, 1, . . .,  − 1 (10) where the state () represents an element of the space   , k= , ,. .., N-; () is input variable which belongs to the space   , k= , ,. .., N-; and the control variable () is subjected to the constraint conditions which are reliant on the current state () for all () ∈   , k= , ,. .., N-, N.
The cost function is expressed as where   indicates the cost function of each time step.Given an initial state (0), the problem lies in seeking a solution set of () that minimizes the cost function.Here, the total amount of energy recovery is selected as cost function.In case that the drive cycle is determined, the vehicle speed at each time is also determined.By dividing the total time into  segments with the time step Δ, then the energy recovery is obtained as follows: where () represents the pressure of accumulator at time k; Δ is referred to as the volume change of gas, which is equivalent to the volume change of oil; () is defined as the flow of hydraulic oil per unit time; () denotes the displacement of pump/motor at time k; () indicates the ratio of the torque coupler at time k.
The cost function at each step is described as follows.
=  ()  () Δ According to Boyle's law, the pressure of the accumulator is defined as follows.
Maximum energy recovery of PHHS is delivered by satisfying the following conditions at each time step: where the displacement of the pump/motor () and the ratio of the torque coupler () are selected as the decision variables.
The gas volume of accumulator is defined as a state variable, such that Three constraint conditions are taken into consideration as follows: (1) The braking torque of PHHS output is less than the torque required by the vehicle.
(2) The speed of the hydraulic pump is no greater than the rated speed of the pump.
(3) The displacement of the hydraulic pump is no greater than the rated speed of the pump.
The method of torque evaluation provided by PHHS is where the () is referred to as the output torque of PHHS at time k,  1 is defined as the efficiency of the hydraulic pump, and   () indicates the braking torque required by the vehicle at time k.
The displacement limitation on the hydraulic pump is where   represents the rated displacement of the pump.The restriction on rotating speed is where () indicates the rotating speed of the flywheel, () is referred to as the velocity of the vehicle,  denotes the tire radius, and   is expressed as the rated rotating speed of the pump.
According to the principle of Bellman, the decision variables under constraint conditions were selected in two urban drive cycles.The displacement of the selected hydraulic pump ranged from 0.02 to 0.05 L/r, with 20 different values.The value of the ratio is either 2.5 or 5.The flow chart of DPbased optimization of energy recovery efficiency algorithm is illustrated in Figure 2.
Vehicle speed in UDDS cycle and 1015 cycle are indicated in Figure 3.
Subsequently, the optimal scheme for the displacement of pump and ratio of the torque coupler in 1015 cycle and UDDS cycle is derived in Figures 4 and 5, respectively.

Simulation and Experiment Results
In this section, two main steps are involved to validate the proposed approach.Firstly, a PHHS Matlab/Simulink model is accurately obtained by repeated comparison with the bench tests.Then, the whole parallel hydraulic hybrid vehicle model by integrating the PHHS Simulink model into the ADVISOR is utilized for verification of the proposed approach.
. .Test Bench Experiments.Further with the mathematical model of the major components for PHHS, a simulation model is established as shown in Figure 6.The simulation model is categorized into three separate subsystems, i.e., flywheel, torque coupler, and hydraulic hybrid power system.The flywheel provides the power source of the system, which is designed to drive the hydraulic system by the torque coupler to store the energy in the accumulator.Meanwhile, the hydraulic system provides the brake torque for the flywheel.
The test bench for the PHHS is built up in the lab as illustrated in Figure 7, which is intended to verify the simulation model.The main parameters in relation to the test bench are listed in Table 1.
As displayed in Figure 7, the flywheel is driven by the motor when the electromagnetic clutch is connected.Then the rotating flywheel is used to simulate the kinetic energy prior to the vehicle being braked.Where the flywheel reaches a certain speed, the electromagnetic clutch (2) is disconnected, the electromagnetic clutch (7) is connected, and the hydraulic pump/motor is driven by the flywheel.At this moment, the hydraulic pump/motor works as a pump to fill the accumulator with oil.The kinetic energy of the flywheel is converted into a hydraulic potential energy stored in the accumulator.When the energy is released by PHHS, the dump valve is opened up, and the hydraulic potential energy in the hydraulic accumulator drives the pump/motor.At this moment, the motor/pump drives the flywheel to rotate in the form of a motor, converting the hydraulic potential energy into kinetic energy.The magnetic powder brake is utilized to supplement the braking force when the braking torque provided by the hydraulic regenerative braking system is insufficient.The wheel motor displayed in the Figure 7 is purposed for conducting future research, for which it is not involved in this study.
Over the course of the simulation and the bench test experiment, the initial pressure of the hydraulic accumulator is set at 5 MPa.According to the empirical formula, the  minimum working pressure of hydraulic accumulator is 5.6MPa, and the displacement of hydraulic pump is 0.05L/r.The initial speed of flywheel is chosen as 360 rpm, 300 rpm, and 300 rpm, which correspond to the friction braking torque provided by magnetic powder brake with 0Nm, 0Nm, and 40 Nm, respectively.The results from carrying out the simulation and experiment with the flywheel speed as well as the accumulator pressure have been shown in Figures 8 and 9, respectively (the dotted line represents the simulation result, and the solid line represents the experimental result).As revealed by Figure 8, the simulation and experimental results of the flywheel speed are broadly consistent except for some insignificant vibration in the experimental results.At the same time, it is discovered from Figure 9 that the simulation and experimental results of the accumulator pressure are basically unchanged except for the time delay at the initial stage.Due to the impact made by the hydraulic and mechanical components, the time delay of the bench experiment is about 0.7s, and the error between the simulation and the bench experiment comes to approximately 10%.The greater the initial speed of the flywheel, the longer the duration of the whole process.As the initial speed of the flywheel is on the rise, the final pressure of the hydraulic accumulator is also showing an increasing trend.The energy recovery efficiency in terms of each initial condition is calculated according to (12), which is already indicated in Table 2.When the mechanical friction is involved, the ultimate pressure of the hydraulic accumulator is lowered, and the energy recovery efficiency of the hydraulic regenerative braking is reduced as a result.Moreover, it is visible as in Figure 9 that the final pressure of the hydraulic accumulator is slightly lower than that shown in the simulation results.This is attributed to the energy loss from the mechanical components and hydraulic pipes in the test bench, which contributes to the ultimate       10.In case the regenerative braking is required, the brake model of the PHHS starts to operate, and the pump/motor operates in the state of the pump to store the energy into the hydraulic accumulator.When the vehicle starts or accelerates, the drive model of PHHS begins to work, the pump/motor works in the state of the motor, and the hydraulic accumulator releases the stored energy.The torque exerted by the PHHS acts on the vehicle motion model.The vehicle controller determines the vehicle input torque with a comparison of the difference between the actual running speed of the vehicle and the desired speed from the drive cycle.
The displacement of the hydraulic pump and the ratio of the torque coupler are fixed without applying DP algorithm.The selection of displacement and transmission ratio is determined based on the following rules.The displacement of the hydraulic pump is 0.02L/r, and the ratio of the torque coupler is 2.5 under the 1015 cycle condition.The displacement of the hydraulic pump is 0.035L/r, and the ratio of the torque coupler is 2.5 in the UDDS cycle condition.With respect to the proposed optimization of energy recovery  11.The detailed vehicle parameters are listed in Table 3.
The simulation results derived from Figures 12 and 13 demonstrate that the optimal strategy of the pump displacement and the torque coupler transmission ratio based on the DP algorithm is effective.As compared to the PHEV without DP, the efficiency of braking energy recovery is enhanced with the increased final pressure of the increasing accumulator and the regenerative braking torque of PHHS.Compared with the two drive cycles, the frequency of braking is lower, the strength is smaller in 1015 drive cycle, and the maximum speed of the vehicle stands at 43mph.In contrast, the braking frequency is higher, the braking strength is bigger in UDDS drive cycle, and the maximum speed of the vehicle reaches 56 mph.The amount of electricity consumed by the vehicle in two different drive cycles is described in Table 4.
From Table 4, it is known that the vehicle makes a 5.4% saving on the electric amount using the DP algorithm in 1015 drive cycle and a 6.5% saving on UDDS drive cycle compared with the PHEV without DP algorithm.

Conclusions
This paper presents a DP-based optimal energy recovery efficiency algorithm for PHHS.Global optimal solution of displacement and transmission ratio under the known urban drive cycles is obtained by taking the DP approach, where the cost function is defined as the total amount of energy recovery, and the displacement of the pump and the transmission ratio of the torque coupler as the decision variables.The PHHS bench test is performed to verify the PHHS Simulink model.Then, the whole parallel hydraulic hybrid vehicle model is established by integrating the PHHS Simulink model into the ADVISOR.Finally, the improved performance in terms of energy recovery efficiency for the proposed approach is validated.This performance improvement thus will lead to a significant reduction in fuel consumption when driving on the road.Future work will be dedicated to the design of the implementable strategy of brake energy recovery for a real parallel hydraulic hybrid bus based on the results in this paper.

Figure 4 :Figure 5 :
Figure 4: Optimal solution of the displacement of pump and ratio of the torque coupler from 1015 drive cycle.

Figure 8 :
Figure 8: Simulation and experimental results of flywheel speed.

Final 6 Figure 9 : 8 Mathematical
Figure 9: Simulation and experimental results of accumulator pressure.

Figure 10 :
Figure 10: Structure diagram of the whole vehicles.

Figure 11 : 8 Figure 12 :
Figure 11: The electric vehicle model combined with the proposed PHHS model.

Table 1 :
Parameters of the test bench.

Table 2 :
The final pressure of accumulator and energy recovery efficiency at different conditions.

Table 4 :
The amount of electricity consumed.