Variable Impedance Control for Bipedal Robot Standing Balance Based on Artificial Muscle Activation Model

The bipedal robot should be able to maintain standing balance even in the presence of disturbing forces. The control schemes of bipedal robot are conventionally developed based on system models or ﬁxed torque-ankle states, which often lack robustness. In this paper, a variable impedance control based on artiﬁcial muscle activation is investigated for bipedal robotic standing balance to address this limitation. The robustness was improved by applying the artiﬁcial muscle activation model to adjust the impedance parameters. In particular, an ankle variable impedance model was used to obtain the antidisturbance torque which combined with the ankle dynamic torque to estimate the desired ankle torque for robotic standing balance. The simulation and prototype experimentation results demonstrate that the control method improves the robustness of bipedal robotic standing balance control.


Introduction
Nowadays, a vast variety of bipedal robots have been created to help humans and they have been applied to a myriad of social applications, such as military training, medical services, industrial manufacturing, and other fields [1]. In these applications, the working environments are usually unknown, which have the risk to interfere with the standing balance of the bipedal robots. How to prevent robots from falling, that is, standing balance control, is a fundamental control problem for bipedal robots. e ankle plays an important role in bipedal robotic standing balance control [2], which raises concerns for robotic ankle control.
Researchers have done a lot of works on bipedal robotic standing balance control. Vukobratovic et al. [3] referred to the principle of mechanical arm balance control and proposed the Zero Moment Point (ZMP) control method which adjusts the joint torque according to the trajectory of realtime ZMP. However, the ZMP control method has some limitations: the ZMP was calculated by sensor information feedback which lags behind the actual attitude change, and the delay will cause the controller ring [4]. According to the law of conservation of momentum, some scholars simultaneously adjust the angular and linear momentum to complete the bipedal robotic standing balance control [5,6], and this approach relies on the robotic dynamic model which is difficult to improve the robustness of the standing balance control. With the development of intelligent control algorithms which are increasingly being used to solve the robotic standing balance control problems [7], the intelligent control algorithms rely on a large amount of test data. In addition, robots have some hardware limits. For example, the input saturation and actuator dead zones affect the control robustness, and the intelligent algorithms and adaptive methods provide effective solutions [8,9]. e bipedal robots have a complex structure, with the characteristics of system nonlinearity and structural variability, which brings about great challenges to motion control [10]. rough a long period of evolution, human beings have the ability to adapt and swiftly respond to environmental changes. ese abilities of human beings have provided the best guidance to advance the design of robotic controllers. e authors in [11] attempted to build the virtual neuromuscular model for robotic control. is approach can generate human-like diverse and robust locomotion behaviors. However, the major components of this control strategy are twofold: the virtual muscle model and the muscle activation model. However, the virtual model involves many control model parameter sets, such as virtual muscular parameters, which limit the general applicability of this approach [12].
To make robotic joints present a "gloppy" or "springy" compliant control behavior similar to human joints, the concept of impedance control in the field of robots has been proposed by Hogan [13]. Impedance control is extensively employed in robotic control and its robustness and feasibility have been acknowledged by many research studies [14,15]. However, a fixed impedance model may not suffice in many applications, and variable impedance is necessary to achieve optimal performance of the system [16]; for example, human beings have the ability to adjust their joint impedance through muscle contraction.
In this paper, the robotic ankle is streamlined into an impedance model, and an artificial muscle activation model is built to adjust the impedance parameters. en, the variable impedance control based on artificial muscle activation for bipedal robot balance was proposed. Specifically, the ankle antidisturbance torque is obtained by constructing the ankle variable impedance model, and the ankle dynamic torque is calculated by constructing an inverted pendulum model of the bipedal robot. By combination of antidisturbance torque and dynamic torque, the expected ankle torque for standing balance control is estimated. e main contributions of this work are threefold: (1) it proposes a variable impedance control for bipedal robot standing balance, (2) it develops an impedance parameter sets update approach based on artificial muscle activation, and (3) the proposed work was validated and evaluated by both simulation and prototype experimentation approach.

Methods
e proposed variable impedance control method is used to estimate the ankle desired torque (τ q ) for bipedal robotic standing balance control. Accordingly, the proposed variable impedance control method is composed of three vital components: a dynamic model to estimate the dynamic torque (τ r ), an ankle impedance model to calculate the disturbance torque (τ e ), and an impedance adjust component to update the impedance parameters based on artificial muscle activation model.

Control Method Overview.
e framework of the proposed variable impedance control for bipedal robot standing balance is illustrated in Figure 1, with the vehicle platform acceleration and deceleration in this work to simulate various perturbations of a bipedal robot standing balance.
ere are two control loops in parallel, with the dynamic model and the impedance model being the main components of the two loops. e inputs of the dynamic model are the robotic ankle angle and its change rate [θ foot _ θ foot ], and the output is the dynamic torque (τ r ). Likewise, the inputs of the impedance adjust component are the same as those of the dynamic model. e model first calculates the muscle activation (a) based on the artificial muscle activation model and, subject to the parameters update operation, inputs to the impedance model which is based on the robotic ankle and its change rate output the disturbance torque (τ e ). From this, the ankle desired torque (τ q ) for bipedal robotic standing balance control is aggregated output of τ r and τ e .

Dynamic Torque Estimation.
e dynamic model describes the relationship between the motion of the bipedal robot and the dynamic torque (τ r ) of the robotic joint. In order to study the bipedal robotic standing balance ankle strategy, the complex actions such as arm swing, curved body, and step can be ignored. Without loss of generality, the bipedal robot was simplified as an inverted pendulum model with swinging around the ankle, in this paper, and all the robotic weight is concentrated on the center of mass (CoM). e robotic ankle in the initial state is marked as the coordinate origin, and the horizontal and vertical directions are marked as the x-axis and y-axis, respectively. e bipedal robotic inverted pendulum model is illustrated in Figure 2.
Based on the established x-y coordinate system, the differential equation of the robotic torso rotating around the ankle can be expressed as (1) e robotic CoM horizontal force (F x ) can be expressed as that is, e robotic CoM vertical force (F y ) can be described as that is, According to equations (1), (3), and (5), the bipedal robotic standing balance ankle dynamic model can be described as where I represents the rotational inertia, and I � ml 2 .

Disturbance Torque
Estimation. e impedance model refers to the dynamic relationship between the input flow and the output effort at the interaction port between a manipulator and its environment [17]. is paper regarded the robotic ankle joint as an impedance model and used it to estimate the ankle disturbance torque. e bipedal robotic ankle impedance model's schematic diagram is shown in Figure 3.
In the Cartesian coordinate system, the external disturbance (f e ) can be estimated by the ankle impedance model, which can be expressed as where k, b, and m stand for the stiffness, damping, and inertia in the ankle impedance model, respectively, and x e represents displacement errors. e Jacobi matrix J(θ foot ) is defined as where θ foot denotes the bipedal robot ankle angle. e disturbance torque can be an estimation based on the ankle impedance model, which can be expressed as e kinematic relationship between the bipedal robot ankle Cartesian coordinates system and the joint coordinates system is where l is the mass of center vertical height of the bipedal robot.
For the task of bipedal robot standing balance control, the ankle angle θ foot , that is, the robotic swing amplitudes, is usually small; that is, sin θ foot ≈ θ foot . According to equation (3), the disturbance torque can be rewritten as where θ e � θ foot − θ ref denotes the robotic tilt angle and θ ref is the ankle angle as the robot at the equilibrium position. K, B, and M stand for the target stiffness, damping, and inertia value of the robotic ankle impedance model, and the following equations were met: Let u � τ r ; the robotic tilt angle and its time derivative are the state variables for a model system x � (θ e , _ θ e ); then the equation can be written as According to the Zero Moment Point (ZMP) stability criterion theory, if ZMP is inside of the support polygon, the bipedal robot can maintain a stable and upright state; if ZMP is outside of the supporting polygon (including the boundary), the bipedal robot is in an upright unstable state. A cost function can be constructed by the square value of ZMP deviation, which can be defined as According to the above cost function based on ZMP, the linear quadratic regulator (LQR) can be used to address the bipedal robotic standing balance control problem. e optimal solution is determined by Behrman's optimal principle, and the form of the optimal solution can be expressed as where K u represents the appropriate gain parameters, and following equation holds: where P is a unique positive-definite matrix and satisfies the famous Riccati differential equations: where Q is a symmetric positive-definite matrix. From the cost function equation (14), R can be expressed as From equations (15), (17), and (18), we can get In order to avoid the noise effect caused by the quadratic differentiation of ankle angle, the inertia term in the impedance model is ignored in this paper, and the analytical optimal solution of the ankle impedance model is When the bipedal robot suffers from external disturbances, the state variables deviate from the equilibrium point, and the ankle impedance parameters will be changed. Taking the robotic ankle like the human ankle, the ankle muscle contraction causes human ankle impedance parameter sets to be different at different ankle states. In other words, ankle muscle contractions are closely related to the regulation of human ankle impedance. e variable impedance characteristics of the human ankle make that have the advantages of low energy consumption and fast stability.

Parameter Update.
e neuromuscular research suggests muscle activation linked to humanoid joint mechanical impedance. In this paper, the impedance parameters were updated by applying the muscle stretch reflex model which is a fast muscle contraction generation mechanism. e muscle stretch reflex model sensory information is motivated by the signals based on the muscle spindle length change and its contraction velocity [18]; and the muscle spindle length can be gained using the ankle angle θ foot . In particular, the ankle muscle spindle length l m can be expressed as where r foot stands for the attachment radius of the ankle muscle, ρ denotes the scaling factor representing the muscle fiber pennation angle, l opt describes the optimal length of the muscle spindle, l m , at which the muscle can provide the maximum isometric force, θ ref is the ankle reference angle at which l m � l opt , and θ max is a constant ankle angle value. From this, the muscle spindle contraction velocity, v m , can be computed via the time derivative of muscle spindle length value l m . e muscle activation value, a, can be computed using the positive feedback reflex scheme. e muscle activation is equal to the preactivation a 0 plus a feedback component, which can be expressed as where k l denotes the feedback gain for muscle spindle length offset, d v expresses the feedback gain for the muscle spindle contraction velocity, and l 0 represents the muscle spindle length under muscle relaxation. e muscle activation is constrained to the range between 0 and 1.
According to [19], the joint stiffness K can be estimated by multiplication of the intrinsic constant stiffness K 0 and the coordinated muscle cocontraction α, and it can be expressed as e optimal stiffness value in equation (20) was seen as the intrinsic constant stiffness in this paper, and the coordinated muscle cocontraction α can be depicted as where β 1 and β 2 are constant coefficients and p (t) stands for stiffness index which can be identified based on the moving average of the ankle joint muscle activation, and it can be illustrated as follows: where w i is the weight of the muscle activation which is defined as . (26) In previous work, the study suggested a linear relationship between square-root joint stiffness and damping value. erefore, joint damping value can be described as where v is a constant coefficient.

Experimentation
e proposed variable impedance control was applied to a bipedal robot standing balance control on a moving vehicle for system validation and evaluation through the simulation and prototype experimentation approach.

Simulation.
e simulation platform was constructed using the OpenSim software as shown in Figure 4, and all the data processed was using Matlab. e parameters of the bipedal robot module in this simulation are listed in Table 1.

Case 1. Variable Impedance Control Simulation
During the complete simulation implementation process, the movement of the vehicle includes three states: stationary, acceleration, and constant speed. Among them, the vehicle only interferes with the bipedal robot standing balance during acceleration. In this simulation, the acceleration of the vehicle set 0.5 m/s 2 as the disturbance; and the impedance model parameters were calculated by equations (20), (23), and (27). For the robotic body tilt angle, the counterclockwise swing is defined as the negative direction, and the clockwise swing is defined as the positive direction. e simulation results including robotic body tilt angle, the ankle torque, stiffness, and damping value curve are shown in Figure 5.
Before the 5 s time point, the vehicle is stationary, there is no external interference, the robot can quickly adjust to a standing balanced state, and the stabilized equilibrium position is about 2.0°. e vehicle applies 0.5 m/s 2 acceleration at 5 s time point, and the robotic body tilt angle leans forward to about −1.7°and then leans backward to about 2.6°. e bipedal robot gradually stabilized at around 2.0°. e ankle torque presents a tendency to increase first and then decrease, as illustrated in Figure 5(b). During the simulation implementation process, the stiffness and damping values of the ankle impedance model are varying depending on the robotic stationary balance, as shown in Figures 5(c) and 5(d). In the antidisturbance range, the stiffness and damping amplitudes variation is quite large.
In this simulation part, the robustness of the proposed variable impedance control method was verified by conducting multiple simulations with different vehicle acceleration. When the vehicle acceleration is 0.5 m/s 2 , 1.0 m/s 2 , and 1.5 m/s 2 , the results of bipedal robotic body tilt angle and ankle torque are summarized in Figure 6. As the vehicle acceleration increases, the bipedal robotic maximal body tilt angle swing range is increasing. After the standing balance control is completed, the bipedal robot returns to the consistent balance position: within the range of 1.7∼2.0°. e simulation results indicate that  Case 2. Constant Impedance Control Simulation is simulation was a continuation of the first simulation which was used as a comparative simulation to verify the power of the proposed variable impedance control system. In this simulation, the moving process of the vehicle is the same as that discussed in the first simulation; and the impedance model parameters were calculated by equation (20). In order to facilitate the comparison, the robotic body tilt angle and the corresponding ankle torque under the 1.5 m/s 2 vehicle acceleration are illustrated in Figure 7.  From this figure, it can be seen that the change trends of the robotic body tilt angle and the ankle torque have some differences. Compared to the results led by variable impedance control, the tilt angle under constant impedance, in reference to the equilibrium position, is larger, and the regulating process to equilibrium position is longer. Correspondingly, the ankle torque is also larger and the process is longer under the constant impedance, as shown in Figure 7(b). is suggests that the proposed variable impedance control outperforms the constant impedance control in robotic standing balance control.

Prototype Experimentation.
e bipedal robotic prototype was constructed as shown in Figure 8. In this experimentation part, the bipedal robotic prototype was standing on a moving vehicle to verify the effectiveness of the proposed variable impedance control applied to the robotic prototype.
During the complete experimentation implementation process, the movement of the vehicle includes three states: stationary, acceleration, and constant speed. For the vehicle acceleration about 1.0 m/s 2 , the bipedal robotic standing balance process is shown in Figure 9. From left to right, the bipedal robot is tilting backward, tilting forward, and finally returning to the balanced position. e experimentation results of the bipedal robotic standing balance control based on the proposed variable impedance model are shown in Figure 10, which shows the experimental results of two arbitrarily selected standing balance control processes. During the bipedal robotic standing balance control, the ankle impedance model stiffness and damping value are automatically updated according to the changes of the virtual muscle activation amount to adapt to the swing state of the bipedal robotic trunk. e changing process is shown in Figures 10(c) and 10(d). e ankle torque was calculated based on the ankle variable impedance model, the maximum ankle torque was 1.1 Nm, and the minimum ankle torque was −0.5 Nm, as shown in Figure 10 e experimental results demonstrate the power of the proposed control system in the applicability of bipedal robotic standing balance.

Conclusion
Aiming at the problems of poor robustness of commonly used bipedal robotic control methods, this paper proposed a variable impedance control based on an artificial muscle activation model, which was used to generate the desired ankle torque for bipedal robotic standing balance. Specifically, the desired ankle torque was estimated combination of ankle antidisturbance with ankle dynamic torque, the ankle antidisturbance torque is calculated by constructing the ankle variable impedance model, and the ankle dynamic torque is obtained by constructing a bipedal robotic inverted pendulum model. e simulation and prototype experimentation results based on different vehicle acceleration demonstrated the power of the proposed variable impedance control system in improving the robustness of bipedal robotic standing balance control. Although the proposed control system in this paper only targets the robotic ankle

Data Availability
e data used to support the findings of this study are included within the article.

Conflicts of Interest
e authors declare that they have no conflicts of interest.