Biologically Inspired Robotic Arm Control Using an Artificial Neural Oscillator

We address a neural-oscillator-based control scheme to achieve biologically inspired motion generation. In general, it is known that humans or animals exhibit novel adaptive behaviors regardless of their kinematic configurations against unexpected disturbances or environment changes. This is caused by the entrainment property of the neural oscillator which plays a key role to adapt their nervous system to the natural frequency of the interacted environments. Thus we focus on a self-adapting robot arm control to attain natural adaptive motions as a controller employing neural oscillators. To demonstrate the excellence of entrainment, we implement the proposed control scheme to a single pendulum coupled with the neural oscillator in simulation and experiment. Then this work shows the performance of the robot arm coupled to neural oscillators through various tasks that the arm traces a trajectory. With these, the real-time closedloop system allowing sensory feedback of the neural oscillator for the entrainment property is proposed. In particular, we verify an impressive capability of biologically inspired self-adaptation behaviors that enables the robot arm to make adaptive motions corresponding to an unexpected environmental variety.


Introduction
Recently biologically inspired systems and control methods have been studied widely, in particular in robotics field.Thus, a number of virtual human or animal-like robots and control approaches have been yielded for the last decade.Owing that such approaches enable robots to embody autonomous dynamic adaptation motion against unknown environmental changes, its attraction has become generally gained and issued.This is because the musculoskeletal system is activated like a mechanical spring by means of the central pattern generators CPGs and their entrainment property 1-3 .The CPGs consist in the neural oscillator network and produce a stable rhythmic signal.Entrainment of the neural oscillator plays a key role to adapt the nervous system to the natural frequency of the interacted environments incorporating a sensory feedback.Hence, the neural oscillator in the nervous system offers a potential controller, since it is known to be robust and have an entrainment characteristic as a local controller in humans or animals.
Relating these previous works, the mathematical description of a neural oscillator was presented in Matsuoka's works 1 .He proved that neurons generate the rhythmic patterned output and analyzed the conditions necessary for the steady state oscillations.He also investigated the mutual inhibition networks to control the frequency and pattern 2 , but did not include the effect of the feedback on the neural oscillator performance.Employing Matsuoka's neural oscillator model, Taga et al. investigated the sensory signal from the joint angles of a biped robot as feedback signals 3, 4 , showing that neural oscillators made the robot robust to the perturbation through entrainment.This approach was applied later to various locomotion systems 5-7 .In addition to the studies on robotic locomotion 8 , more efforts have been made to implement the neural oscillator to a real robot for various applications.Williamson showed the system that had biologically inspired postural primitives 9 .He also proposed the neuromechanical system that was coupled with the neural oscillator for controlling its arm 10 .Arsenio 11 suggested the multiple-input describing function technique to evaluate and design nonlinear systems connected to the neural oscillator.
As above, existing works in field of biologically inspired system based on neural oscillators have yielded notable results in many cases.However approaches for a proper behavior generation and complex task were not clearly described due to the difficulty in application considering a real robotic manipulator coupled with the neural oscillator.Yang et al. has presented simulation and experimental results in controlling a robot arm and humanoid robot incorporating neural oscillators 12-15 .Apart from such the proposed parameter optimization method, we newly address an intuitive and efficient approach for a desired task of the neural-oscillator-based control.In addition, this work addresses how to control a real system coupled with the neural oscillator for a desired task.For this, real-time feedback is implemented to exploit the entrainment feature of the neural oscillator against unpredictable disturbances.
In the following section, a neural oscillator is briefly explained and its entrainment property is described and verified.Details of the dynamic stability of the developed methodology are discussed in Section 3. The experimental results are presented in Sections 4 and 5. Finally, conclusion is drawn in Section 6.

Matsuoka's Neural Oscillator
Matsuoka's neural oscillator consists of two simulated neurons arranged in mutual inhibition as shown in Figure 1 1, 2 .If gains are properly tuned, the system exhibits limit cycle behaviors.The trajectory of a stable limit cycle can be derived analytically, describing the firing rate of a neuron with self-inhibition.The neural oscillator is represented by a set of

Extensor neuron
Flexor neuron nonlinear coupled differential equations given as where x e f i is the inner state of the ith neuron which represents the firing rate; v e f i represents the degree of the adaptation, modulated by the adaptation constant b, or selfinhibition effect of the ith neuron; the output of each neuron y e f i is taken as the positive part of x i , and the output of the whole oscillator as Y out ; w ij 0 for i / j and 1 for i j is the weight of inhibitory synaptic connection from the jth neuron to the ith neuron, and w ei , w fi are also weights from the extensor neuron to the flexor neuron, respectively; w ij y i represents the total input from the neurons inside the network; the input is arranged to excite one neuron and inhibit the other, by applying the positive part to one neuron and the negative part to the other; T r and T a are time constants of the inner state and the adaptation effect of  the ith neuron, respectively; s i is the external input, and g i indicates the sensory input from the coupled system which is scaled by the gain k i .
Figure 2 shows a simple mechanical system connected to the neural oscillator.The coupled method enables a robot to adapt to changing conditions.The desired torque signal to the ith joint can be given by where k oi is the stiffness of the joint, b i is the damping coefficient, q i is the joint angle, and q doi is the output of the neural oscillator that produces rhythmic commands of the ith joint.The neural oscillator follows the sensory signal from the joints; thus the output of the neural oscillator may change corresponding to the sensory input.This is what is called "entrainment" that can be considered as the tracking of sensory feedback signals so that the mechanical system can exhibit adaptive behavior interacting with the environment.

Entrainment Property of Neural Oscillator
Generally, according to Matsuoka's work 1, 2, 16 , the entrainment can be realized under stable oscillation conditions of the neural oscillator.For stable oscillations, if tonic input exists, then T r /T a should be in the range 0.1 ∼ 0.5, for which the natural frequency of the oscillator is proportional to 1/T r .The magnitude of the output signal also increases as the tonic input increases.T r and T a have an effect on control of the delay time and the adaptation time of the entrained signal, respectively.Thus, as these parameters decrease, the input signal is well entrained.And the minimum gain k i of the input signal enlarges the entrainment capability, because the minimum input signal is needed to be entrained appropriately in the range of the natural frequency of an input signal.In this case, regardless of the generated natural frequency of the neural oscillator and the natural frequency of an input signal, the output signal of the neural oscillator locks onto an input signal well in a wide range.Figure 3 illustrates the entrainment procedure of the neural oscillator.If we properly tune the parameters of the neural oscillator, then the oscillator exhibits the stable limit cycle behaviors.In Figure 1, the gain k of the sensory feedback was sequentially set as 0.02, 0.2, and 0.53 such as in Figures 3 a ,

Verification of Entrainment Property through Experiment
In this subsection, we experimentally verify the entrainment capability of the neural oscillator and its validation addressed in above Section 2.2. Figure 4 shows the experiment setup to implement entrainment to real robotic systems.As illustrated in Figure 2, the single pendulum is tightly coupled with the neural oscillator.This means that the neural oscillator observes and entrains the encoder value of the motor in terms of the sensory feedback and    the output of the neural oscillator drives the motor directly.Hence, the pendulum is excited periodically by the output generated in the neural oscillator.And also the coupled oscillatorpendulum exhibits natural adaptive motions even though we swing the pendulum arbitrary 22 s to 38 s and 56 s to 74 s sequentially as shown in Figure 5.It can be confirmed from the experimental result that entrainment of the neural oscillator enables the coupled system to show naturally self-adapting motions against unpredictable disturbances.

Control Scheme Based on Neural Oscillator
The neural oscillator is a nonlinear system; thus it is generally difficult to analyze the dynamic system when the oscillator is connected to it.Therefore we studied the existence of singular points and their stability of the neural oscillator in time domain analysis investigating equilibrium states 16 .Also analysis of a simple mechanical system coupled with the neural oscillator through a graphical approach known as the describing function analysis has been proposed earlier 17 .The main idea is to plot the system response in the complex plane and find the intersection points between two Nyquist plots of the dynamic system and the neural oscillator.The intersection points indicate limit cycle solutions.However, even if a rhythmic motion of the dynamic system is generated by the neural oscillator, it is usually difficult to obtain the desired motion required by the task.This is because many oscillator parameters need to be tuned, and different responses occur according to the interoscillator network.Hence, we propose the control method that enables a robot system to perform a desired motion without precisely tuning parameters of the neural oscillator within the range of its well-known stable condition.
Figure 6 illustrates a schematic model of a robot arm whose each joint is coupled to the neural oscillators.And a virtual force leads the coupled robot arm to a given motion.The virtual force inducer VFI such as springs and dampers which is supposed to exist virtually at the end-effector of a manipulator can be transformed into equivalent torques.This causes the end-effector of a robot arm to draw according to the desired trajectory calculating position error.Also, it is shown that ill-posedness of inverse kinematics can be resolved in a natural way without using any artificial optimization criterion 12-14 .However, even in such a method, kinematic configurations including redundant joints may not be guaranteed, even though the posture of a robot arm could be set only within a certain boundary.
From this point of view, it would be advantageous if neural oscillators are hardly coupled to each joint of a robot arm.When the oscillators are implemented to a robotic arm, Mathematical Problems in Engineering they provide a proper motor command considering the movements of the joints with sensory signals.Since biologically inspired motions of each joint as described in Section 2 are attained by entrainment of the neural oscillator, the coupled joint can respond intuitively with the entrainment property according to environmental changes or unknown disturbance inputs performing an objective motion.In addition, each neural oscillator can be tuned in order to give the criterion for limitation of motion within a driving range to the joints considering the amplitude of the sensory feedback signal.
In general, dynamics of a robot system with n DOFs could be expressed as H q q 1 2 Ḣ q S q, q q g q u, 3.1 where H denotes the n × n inertia matrix of a robot, the second term in the left-hand side of 3.1 stands for coriolis and centrifugal force, and the third term is the gravity effect.Then a control input for a rhythmic motion of the dynamic system shown in 3.1 is introduced as follows: where where k and ς 0 are the spring stiffness and damping coefficient, respectively, for the virtual components.C 0 is the joint damping.k o and q odi are the stiffness gain and the output of the neural oscillator that produces rhythmic commands, respectively.The control inputs as seen in 3.2 consist of two control schemes.One is based on Virtual spring-damper Hypothesis 18, 19 and the other is determined in terms of the output of the neural oscillator as illustrated in 2.2 .In the control input of 3.2 , the first term describes a joint damping for restraining a certain self-motion which could occurr in a robot system with redundancy, and the second term means PD control in task space by using of Jacobian transpose, and also a spring and a damper in the sense of physics.Appropriate selection of the parameters such as joint damping factors C 0 , stiffness k, and damping coefficient ς renders the closed-loop system dynamics convergent, that is, x is converged into x d and both of ẋ and q become 0 as time elapses.In general, the neural oscillators coupled to the joints perform the given motion successively interacting with a virtual constraint owing to the entrainment property if gains of the neural oscillator are properly tuned 13, 16 .In the proposed control method, the VFI is considered as a virtual constraint.Also, the coupled model enables a robotic system to naturally exhibit a biologically inspired motion employing sensory signals obtained from each joint under an unpredictable environment change.

Experimental Verifications
For considering the possibility of the proposed control scheme described in Section 3, a real robot arm with 6 degrees of freedom see Figure 7 b is employed and a real-time control system is constructed.This arm controller runs at 200 Hz and is connected via IEEE1394 for data transmission at 4 KHz.ATI industrial automation's Mini40 sensor was fitted to the wrist joint of the arm to detect external disturbances.The appropriate parameters in Table 1 were used for the neural oscillator.Also Table 1 illustrates the parameters on the arm dynamics of the real robot.The initial parameters of the neural oscillators were basically selected under the stable oscillation condition of the neural oscillator by investigating the dynamic response of the neural oscillator.The stable oscillation condition of the population of the connected unit neuron is well described in the Matsuoka's works 1, 2 .And then, the amplitude and period of the neural oscillator's output for generating the desired input are determined by controlling the parameter s tonic input and T r /T a time constants , respectively.Figure 8 shows the arm kinematics of the real robot arm.Since the desired motions are generated in the horizontal plane, q 1 and q 3 are set to 90 • .The initial values of q 5 and q 6 are set to 0 • , respectively.q 2 and q 4 , corresponding to θ 1 and θ 2 in Figure 7 a , respectively, are controlled by the neural oscillators.
Various tasks in cases 1 through 3 of Figures 9 a , 9 b , and 9 c are verified with respect to adaptive motion of the arm against arbitrary forces.As drawn in Figure 9, the end-effector of the two-link robot arm follows the desired trajectories dashed lines .Then while the robot arm shows the desired motion, arbitrary external forces are applied to the end-effector of the robotic arm according to the direction of the arrows indicated in Figures 9 a , 9 b , and 9 c .Figures 10 a to 10 c indicate experimental results on case 1 to case 3 of Figure 9, respectively.Through these cases, we examine whether various desired motions such as motionless status as well as linear and circular motions can be attained or not.Basically kinds of these motions were verified from the results of Figure 10.In Figure 10, the dotted lines in the center part of the figure show the desired motions and overlapping lines illustrate the motion trajectories that are drawn in terms of the end-effector of the real robot arm.In addition, we pushed and pulled the end-effector along the positive and negative x direction as shown in Figure 10 a .And such conditions were also applied to the robot arm along the y direction in order to evaluate an adaptive feature of the proposed control method under additive external disturbances.It can be verified from the experimental result of Figure 10 a that the robot arm is moved well according to the direction of the applied force about 10 N and below .If an arbitrary force exists, then it follows that the end-effector of the robot arm shows a compliant motion even in the linear motion and circular motion of the robot arm as seen in Figures 10 b and 10 c .The force and torque F/T sensor value in the x and y direction are measured and calculated into each joint value.By this, the joint angles are changed according to the direction of the impact of the force induced by the collision, which makes the neural oscillators entrain the joint angles for biologically inspired motion.Hence a change in the output produced intuitively from the neural oscillator causes a change in the joint torque.Finally the joint angles are modified adequately.Thus, it can be confirmed that the proposed neural-oscillatorbased robot arm control approach successfully dealt with unexpected collisions sustaining desired motions.Figure 11 conceptually illustrates the objective tasks with experimental setup for the validation of the proposed control scheme.We evaluate the entrainment capability of the neural oscillator that enables a manipulator to implement and sustain the given task under various environmental changes.Hence, in order to verify the possibility of such adaptation performance, we apply various circumstances to the coupled oscillator-robot arm with the tasks with respect to opening and closing a drawer as seen in Figure 11.We tightly joined the end-effector of the robot arm to the drawer.The end-effector's direction of the robot arm is designed in accordance with the direction to open or close the drawer under the condition that the drawer is not rotated but fixed.In Figures 11 b and 11 c , the drawer was rotated clockwise and counter-clockwise about 12 • for considering unknown environmental changes.Then, the end-effector of the robot arm brings about various collision problems with the drawer due to a different direction between the end-effector of the robot arm and the drawer.Now, we will examine what happens in the arm motion on performing the objective task if additive external disturbances exist.

Experimental Results
Figures 12 a and 12 b illustrate the experimental results on each joint output of the robot arm as the sensory feedback of the neural oscillator is turned off and on, respectively.In the first case of Figure 11 a , the desired motion of robot arm is not changed owing that the drawer is immovable during 0 s to 20 s.The first joint q 2 and the second one q 4 are actuated to move to the distance corresponding to an external force as explained in above Section 3. Hence, if the drawer rotates about 12  1 and 2 that is changed in terms of forces applied when the drawer is rotated.Comparing the result of Figure 12 a with the neural-oscillator-based control see Figure 12 b if the sensory information is fed again, it can be observed that the outputs of each joint and neural oscillator are changed whenever unknown disturbances are induced into the robot arm.Such effect could be accomplished owing that the oscillator-based control reproduces the desired joint input entraining the joint motion coupled with the neural oscillator through sensory feedback.In Figure 12 a , the output of the neural oscillator sustains a certain oscillation because the sensory feedbacks were turned off and the joint desired input is slightly changed in terms of the torque sensor feedback in comparison with Figure 12 b .Figure 13 shows the snap shots of the robot arm controlled when sensory information is not fed again.The snap shots in Figure 14 show the motion of the robot arm implementing the proposed control approach based on the neural oscillator, where we can observe that the end-effector traces the rotated drawer direction.
As shown in Figures 15 a and 15 b , the end-effector of the robot arm draws the trajectories corresponding to the desired motion for opening and closing the drawer.The straight dotted lines indicate the desired trajectories of the robot arm generated by simulation.The blue lines show the trajectories measured at the end-effector of the robot arm in experiments related with Figures 13 and 14.In Figure 15 a , movements of the robot arm are identical with the expected performance although there are inefficient motions due to unknown disturbances.This is because the desired input of each joint is modified adequately by the impedance model measuring external forces with the F/T sensor even though sensory information of the neural oscillators is not fed again.In comparison with this, the individual trajectories drawn by the robot arm in Figure 15 b are completely consistent with the rotated direction of the drawer.Thus, the robot arm coupled with the neural oscillator exhibits the superior potential in adaptive motion exploiting the sensory feedback of the neural oscillator for the capability of entrainment.From experiment results of Figures 12 b , 14, and 15 b , the measured trajectories and movements of real robot arm imply that the neural oscillator enables the robot arm to exhibit the selfadapting motion to enhance adaptive motion sustaining the objective task and motion stability.

Conclusion
We have presented a control scheme for technically achieving a biologically inspired selfadapting robotic motion.In contrast to existing works that were only capable of rhythmic pattern generation for simple tasks, our approach allowed the robot arm to precisely trace a trajectory correctly through entrainment.With this, the proposed method is verified through more complex behaviors of the real robot arm under unknown environmental changes.Also our approach causes appropriate desired motions irrespective of precisely modelling with respect to external disturbances.For such reason, it was observed from experimental results that the novel adaptive motions corresponding to an external force appear clearly.This approach will be extended to a more complex task toward the realization of biologically inspired robot control architectures.

Figure 2 :
Figure 2: Mechanical system coupled to the neural oscillator.

Figure 3 :
Figure 3: Simulation results on the entrainment property of the neural oscillator in cases that k 0.02 a , k 0.2 b , and k 0.53 c , respectively.The solid line is the output of the neural oscillator and the dashed line indicates the sensory signal input.
3 b , and 3 c .When k is 0.02, the output of the neural oscillator cannot entrain the sensory signal input as shown in Figure 3 a .The result of Figure 3 b indicates the signal partially entrained.If the gain k is properly set as 0.53, the neural oscillator produces the fully entrained signal as illustrated in Figure 3 c in contrast to the result of Figure 3 b .

Figure 4 :Figure 5 :Figure 6 :
Figure 4: Experimental setup for driving the single pendulum coupled with the neural oscillator.This operating system runs at 200 Hz in real time.

Figure 7 :
Figure 7: a Schematic robot arm model and b real robot arm coupled with the neural oscillator for experimental test.

Figure 8 :Figure 9 :
Figure 8: Schematic figure for D-H parameter of the robot arm.

Figure 10 :
Figure 10: The trajectories drawn by the end-effector of the real robot arm.

Figure 11 :
Figure 11: Schematic figure on the experiments that robot arm opens and closes a drawer repeatedly.a Fix the drawer in accordance with the robot arm motion, b rotate the drawer clockwise about 12 • , and c rotate the drawer counter-clockwise about 12 • .

Figure 12 :
Figure 12: The outputs of each joint and neural oscillator as the sensory feedbacks of the neural oscillators are turned off a and turned on b .

Figure 13 :
Figure 13: Snap shots of the robot arm motions when sensory information is not fed again in cases of 0 • a , −12 • b and 12 • c rotation of the drawer.
Figures 12 a and 12 b illustrate the experimental results on each joint output of the robot arm as the sensory feedback of the neural oscillator is turned off and on, respectively.In the first case of Figure11a , the desired motion of robot arm is not changed owing that the drawer is immovable during 0 s to 20 s.The first joint q 2 and the second one q 4 are actuated to move to the distance corresponding to an external force as explained in above Section 3. Hence, if the drawer rotates about 12 • and −12 • as illustrated in Figures 11 b and11 c during 20 s to 40 s and 40 s to 60 s, then the robot arm's motion is autonomously altered.In Figures 12 a and 12 b , the blue lines indicate the desired trajectories produced by means of the neural oscillators for the joints 1 and 2. The red dotted lines are the output of the joints

Figure 14 :
Figures 12 a and 12 b illustrate the experimental results on each joint output of the robot arm as the sensory feedback of the neural oscillator is turned off and on, respectively.In the first case of Figure11a , the desired motion of robot arm is not changed owing that the drawer is immovable during 0 s to 20 s.The first joint q 2 and the second one q 4 are actuated to move to the distance corresponding to an external force as explained in above Section 3. Hence, if the drawer rotates about 12 • and −12 • as illustrated in Figures 11 b and11 c during 20 s to 40 s and 40 s to 60 s, then the robot arm's motion is autonomously altered.In Figures 12 a and 12 b , the blue lines indicate the desired trajectories produced by means of the neural oscillators for the joints 1 and 2. The red dotted lines are the output of the joints

Figure 15 :
Figure 15: Trajectories of the end-effector of the robot arm in case that the sensory feedbacks are turned off a and turned on b .

Table 1 :
Parameters of the neural oscillator and robot arm model.