Robot control using electromyography ( EMG ) signals of the wrist

The aim of this paper is to design a human–interface system, using EMG signals elicited by various wrist movements, to control a robot. EMG signals are normalized and based on joint torque. A three-layer neural network is used to estimate posture of the wrist and forearm from EMG signals. After training the neural network and obtaining appropriate weights, the subject was able to control the robot in real time using wrist and forearm movements.


INTRODUCTION
Muscle contraction is the functional unit of body motion and posture control.Coordinated contractions by several muscles allow for various movements of the musculoskeletal system.There are six specific movements associated with the wrist and forearm.Those movements are as follows: • Flexion Bending the joint resulting in a decrease of angle; moving the palm of the hand toward the front of the forearm.

• Extension
Straightening the joint resulting in an increase of angle; moving the back of the hand toward the back of the forearm.

• Adduction (ulna deviation)
Medial movement toward the midline of the body; moving the little finger side of the hand toward the medial side of the forearm.

• Abduction (Radial deviation)
Lateral movement away from the midline of the body; moving the thumb side of the hand toward the lateral side of the forearm.

• Pronation
Internal rotation of the forearm resulting in the palm moving posteriorly, or down.

• Supination
External rotation of the forearm resulting in the palm moving posteriorly, or up.
These movements are controlled by the central nervous system (CNS).The CNS activates the muscles needed to perform a musculo-skeletal movement, and this phenomena can be observed using electromyography (EMG).These NASA researchers have used EMG signals as a substitute for mechanical joysticks and keyboards (Bluck 2001).In this system, the command was corresponded to the gesture for controlling the joystick.This kind of systems were developed in Biomedical engineering field such as prosthetic arm.Also the BioMuse has been demonstrated as an interface to music synthesizers (Atau 1993).Biomuse translated the EMG signals to MIDI signals directly.These two methods are the other extreme.We propose the intermediate systems for using EMG signals.The relationship between EMG activity and the resulting movement has been studied (Koike and Kawato 1995, Mori et al 1992, Koike and Kawato 1994).Previous studies investigated the duration, magnitude, and timing of phasic EMG bursts in relation to movement amplitude, duration, and maximum speed.

EMG signals and "quasi-tension"
In our estimation of arm posture in 3D space, we found that low-pass-filtered EMG signals reflected the firing rate of α motor neurons.We called the signals "quasi-tension" due to their similarity to true muscle tensions.The relationship between the EMG input signal (EMG) and the quasitension output signal ( T) can be represented as a finite impulse response (FIR) filter: (1)

Joint-torque and moment arm
Joint-torque is determined by the product of muscle tension and "moment arm".Moment arm is defined as the distance between the joint axis and the force action line of the muscle.Joint-torque is produced by the difference between agonist and antagonist muscle torques, which depends on muscle tension and moment arm.Joint-torque can be defined as: where τ i is the observed torque for each movement i (flexion, abduction, etc), T j is the quasi-tension of each muscle j, and α i j is the moment arm.We used normalized quasitension represented by "a i j T j ".
Three-layer artificial neural network for determining "equilibrium postures" A three-layer neural network was used to identify the input EMG signals associated with each wrist/forearm movement, i.e., equilibrium posture.The first layer of the neural network consists of the EMG input signals from each muscle.The second layer is the middle layer, and third layer is the output equilibrium posture for each joint.
While the arm is controlled to take a specific posture in 3D space, only the force of gravity affects the arm if there is no external force.The equilibrium between muscle forces and gravitational forces for any joint can be described by the following equation using joint angle θ (n-dimensional vector) and motor command u (k-dimensional vector): where τ m represents the joint torque exerted by the muscles, τ g represents the joint torque generated by the force of gravity, g represents the acceleration of gravity, and h is a nonlinear function which determines the torque generated by the force of gravity from posture.This function can be relatively and easily computed from kinematics knowledge of the arm.Because both τ m and h are nonlinear in θ, the equilibrium position as a solution to equation ( 3) cannot be solved analytically.But the mapping from u to θ is a many-to-one mapping.So a neural network model can be used to determine this relationship.One healthy subject, 24 years of age, participated in this study.Joint torque was measured using a force-torque sensor ( JR3).Simultaneously, EMG signals were recorded using the bipolar configuration.

MEASUREMENT Torque measurement
The subject was seated with his right forearm supported by a fiberglass cuff.The cuff was fixed to the bench at a height comfortable for the subject.The subject's right hand was inserted into a molded plastic grip.The forcetorque sensor was attached to the grip at the center of each of the degree axes of motion.The design of the grip allowed for multiple mounting points for the sensor.The cuff and grip apparatus served several purposes: 1. kept the subject's hand and forearm at a fixed position, maintaining consistent readings 2. kept the subject's hand at a zero-degree position in reference to his forearm; this prevented deviations in joint angle that would affect the direction and magnitude of the force readings 3. allowed the subject to rest his arm and hand; this prevented muscle activations that would have occurred if the subject had to support the weight of his forearm and hand (i.e., hold his arm up).This resulted in a zero equilibrium EMG signal.
To measure joint torque, the subject was asked to flex his hand about a single axis, toward switching directions (e.g., abduction, adduction, abduction, adduction, . ..) for 8 s.EMG signals from all muscles were simultaneously recorded.

Joint-torque estimation
Using equation (2), we calculated moment arm α j from the experimental measure of joint-torque τ and quasi-tension T j for each muscle j.
The moment arm α j are shown in Table 1.
With the calculated α j , we can precisely estimate jointtorque for a given quasi-tension extrapolation from an EMG signal.Coefficient correlations between measured and estimated joint torques for abd/add, flex/ext and pro/sup were 0.94, 0.97 and 0.92, respectively.

EMG measurement
Using pairs of silver-silver chloride surface electrodes, EMG activity was recorded for ten muscles, shown in Figure 1, responsible for movement of the wrist and forearm during 5 s.The subject changed the posture for 44 trials.These muscles and their associated movements are  as follows: • Flexion: Flexor carpi radialis, flexor carpi ulnaris, palmaris longus

Neural network training
Each joint angle was estimated from normalized quasitension using an artificial neural network model.
Figure 2 shows the employed three-layer neural network model.For training, we used the data recorded at 3 s intervals (600 samples) for 44 trials.Therefore, 26,400 samples were used for training.
The training was stopped before the error for the test data began to rise (cross validation method).The correlation coefficients for flexion/extension, abduction/adduction and pronation/supination were 0.90, 0.85 and 0.86, respectively.

Robot control
After classification by the neural network, EMG signals were translated into commands for a Sony Corporation of Japan, AIBO Robot.Movements by the subject's wrist and forearm were mimicked by the robot's head and neck, which shared the same 3 degrees of freedom.These commands were transmitted wirelessly from the computer to the AIBO robot.The entire process, from EMG sampling to robot movement, occurred in real time.

SIMPLE METHOD FOR ESTIMATING 2 DEGREE OF FREEDOMS
We confirmed that the 3 degrees of freedom of the wrist were precisely estimated from EMG signals of the ten muscles associated with wrist movement.In order to estimate the posture precisely, we need to measure EMG signals from several muscles.For the human interface, decreasing the number of electrodes improves efficiency, shortening the time spent locating optimal electrode locations, attaching the electrodes and calibrating the EMG equipment.In this way, 2 degrees of freedom of the wrist were estimated using four channels of EMG signals.We used four muscles for flexion/extension and abduction/adduction as follows: The artificial neural network was trained using 23,600 samples.We tested the neural network on another data set, recorded on a different day. Figure 3 shows the estimation results for the test data.The correlation coefficients for   flexion/extension and abduction/adduction were 0.86 and 0.75, respectively.
It takes approximately 10 min for training using a Pentium 4 (1.8 GHz) processor.In order to reduce the training time, we used the same weight files for estimating left wrist posture with the same subject.We also tested the same  weight files for other subjects.The resulting correlation coefficients from these tests are shown in Table 2.

Robot control
After classification by the neural network, EMG signals were translated into commands for the AIBO Robot.As shown in Figure 4, movements by the subject's wrist and forearm were translated to the forward/backward and turning left/right.These commands were transmitted wirelessly from the computer to the AIBO robot.
The entire process, from EMG sampling to robot movement, occurred in real time.We can switch the control mode, which corresponds to the neck movement and the robot movement.When a muscle on the left arm is activated, the mode is changed.Also AIBO has a CCD camera on head and the subject can see the screen on which AIBO's view is projected.We can control the AIBO while watching the screen as shown in Figure 5.

Controlled by the shoulder movement
For a paralyzed person, it may be difficult for measuring the EMG signal from the wrist to the forearm.This technique is also used to apply in another joint, for example neck or shoulder.In this paper, we use two pairs of muscles for 2 degree of freedoms.In this section, we apply this technique to the shoulder movement.After training the neural network, each input signals correspond to the flexor or extensor for the degree of freedom.Movements of the scapula are retraction/protraction and elevation/depression.For each movement, the muscles trapezius/serratus anterior and levator scapulae/trapezius are operated in Figure 6.
Figure 7 shows the estimation result using the same weight parameters for the neural network.
As you can see, estimation result was accurate and enough for controlling the robot.

CONCLUSION
Joint-torque was estimated from EMG signals in all 3 degrees of freedom of the wrist and forearm.Comparisons between estimated and actual measured joint-torque showed high correlations.This supports the accuracy of our joint-torque model described by equation ( 2).
The most effective robot control was achieved using the neural network model.When the subject performed abduction/adduction and flexion/extension movements, the robot responded accurately by mimicking those movements with its head and neck.This proves that the neural network was successful in discriminating the different EMG signals for abduction/adduction and flexion/extension.The robot was also controlled by estimated wrist posture.
We need to measure the posture and EMG signals simultaneously for training.However, this process would be a very difficult job for daily use.We showed the ability of the neural network to estimate the joint angles precisely using different person's weight file.Also, even if we use different parts of the body that have similar degrees of freedom, we can control the robot.These results indicate that the amputee patient also use this system without any measurement and training.
The accuracy of joint-torque estimation by moment arm and quasi-tension, in addition to the successful control of the robot in 2 degrees of freedom using a three-layer neural network model, proves this design to be a viable humaninterface system.

Figure 2
Figure 2 Three-layer neural network model.
Figure 3Neural network estimation results.

Figure 5
Figure 5 AIBO control with CCD camera.

Figure 7
Figure 7 Estimation result for the shoulder movement.