Development of a multi-DOF electromyography prosthetic system using the adaptive joint mechanism

Abstract: This paper describes an electrically powered prosthetic system controlled by electromyography (EMG) signal detected from the skin surface of the human body. The research of electrically powered prosthetic systems is divided into two main subjects. One is the design of the joint mechanism. We propose the use of an adaptive joint mechanism based on the tendon-driven architecture. This mechanism includes mechanical torque–velocity converters and a mechanism to assist the proximal joint torque by distal actuators. The other subject is the recognition of the EMG signal. For the discrimination of many patterns and nonlinear properties of the EMG signal, we propose a controller based on a simple pattern recognition information process. The system also drives 12 servomotors to move the adaptive joint mechanism. In this paper, we show the proposed system and describe the mechanical design of the prosthetic hand. The experimental results show that the electrically powered devices can be controlled using the proposed method. 9 10 11 12 13 14 15 16 17 18 19


INTRODUCTION
The development of robotics provides useful technology for the medical welfare field.As an example of this, we can mention the electrically powered devices that can be used for support in the daily life activities, functional assistance, or even functional substitution, as the case of prosthetic devices.However, we still have some difficulties for the practical use of these devices.One of the major challenges to overcome is the acquisition of the user's intention from his or her bionic signals, to provide with an appropriate control signal for the device.Also, we need to consider the mechanical design issues such as lightweight, small size, and power supply.For the bionic signals, the electromyography (EMG) signal can be used to control these mechanical products, which reflect the muscles motion, and can be acquired from the body surface.Many studies have reported potential uses and difficulties for the EMG signal pattern recognition (Hudgins et al. 1993;Uchida et al. 1993;Farry et al. 1996;Naruse et al. 1999).
Powered prosthetic hands with multiple degrees of freedom (DOF) can imitate the motions of a natural hand and provide with more functionality than the body powered ones (Neal 1993;Sears and Shaperman 1998;Dechev et al. 2001).Some products are already applied for practical use in the medical area (SensorHand Technical Information Booklet 2001).It is significant not only for the medical area but also for robotics and mechanical engineering because it could be a landmark to achieve a humanoid hand.Between industrial robot hands and the externally powered prosthetic hands (e.g., EMG controlled), there is a large difference in specifications.The prosthetic devices are limited in adequate size, weight, appearance, speed, power, and control precision.To fulfill these requirements, the tendon-driven mechanism has been investigated (Hirose and Ma 1991;Ishikawa et al. 2000).The original paper was related with our first prototype.The mechanism was improved from 10 to 12 DOF because we increase the wrist movability in our new prototype.We developed a tendon-driven robot hand with 10 DOF, which was later improved, adding 2 DOF to the wrist using the same tendon-driven technology.This paper proposes a prosthetic hand with a 12-DOF adaptive joint mechanism based on a tendon-driven mechanism.Figure 1 shows a picture of the proposed system.
A machine with many DOF is difficult to control.Also, EMG signal patterns are not stable and are difficult to classify.We need a good interface to control properly such a machine.Figure 2 shows an example of a general EMG signal.The proposed controller for the EMG prosthetic system is based on the On-Line Learning Method developed by Nishikawa et al. (2000).The On-Line Learning Method performs the adaptive function for the controller of the mechanical device.The EMG-controlled device system uses an EMG signal to transmit human intentions to control the mechanical device; however, EMG signal pattern changes over time even if the hand motion is the same (Uchida et al. 1993;Nishikawa et al. 2000).Also, it is affected by environmental temperature, time dependence, electric noise, and individual differences.Because of these difficulties, the On-Line Learning Method is applied to detect the EMG signal pattern suitable for the motion of hand or body.
Chapter 2 shows the background and requirements for the prosthetic hand.Chapter 3 describes the proposed adaptive joint mechanism and experimental results its characteristics.Chapter 4 describes the controller of the EMG prosthetic system and its performance.

PROSTHETIC HAND
The natural forearm consists of five fingers, palm, and wrist joint.Each finger has three joints and 4 DOF.The palm has many joints, but the motive freedom is integrated into one DOF.The wrist joint has 3 DOF.Therefore, to realize the functional level of human hand, the ideal prosthetic hand should have 24 DOF.
However, the prosthetic hand also has physical restrictions, such as weight, size, and power.The prosthesis size should be produced from the small size (for children) to a larger version (for adults).The weight is limited to the weight of a natural hand.It is necessary to reduce the internal pressure in the socket and to reduce the load on the upper arm, which supports the prosthetic hand.On the basis of these considerations, the prosthetic hand must be designed lightweighted.These constraints limit the  (Nishikawa et al. 2000).
functionality of the conventional myoelectric prosthetic hands on the market (SensorHand Technical Information Booklet 2001), which have only 2-4 functions (grasping, hand opening, and wrist rotations (supination/pronation)).
The grasping force of the prosthetic hand should be strong enough to grip a glass of water; this condition requires more than 3.5 kg/cm.Objective values of the closing speed (300 mm/s) and torque are also severe for the prosthetic hand.Certainly, researchers have acquired higher actuation speed and grasping power, as well as more precise controlled robot hand than the human's one (note that it does not contain the planning of the hand motion).However, these improvements in speed and torque do not come without their trade-off in weight and size, making them unusable in prosthetic applications.
On the contrary, current prosthetic hands on the market can achieve only the gripping force up to 100 Nm and 1.3 Hz as maximum open-close frequency for grasping (SensorHand Technical Information Booklet 2001).The power-weight ratio is a problem for the prosthetic hands, because the motor is placed on the base of the hand, which, in order to increase the gripping force, needs to increase the motor torque, resulting in a heavier device.We propose that it is not necessary to provide high speed and high power simultaneously.If the hand mechanism does have a torque converter that will adapt to the torque needs according to the hand task, the mechanism can increase its grip force and actuation speed without incurring in weight increase.
The current prosthesis controllers cannot control the prosthetic hand as precisely as the central nerve system can do.One reason is the controller capacity, where commercial devices rely on 1 or 2 DOF, which is quite limited to the human hand.Another constraint in the interaction with the current prosthetic devices is the lack of feedback other than the visual for the amputee.It is difficult to execute complex tasks without an adequate amount of feedback information.
From the above discussion, trading off the advantage of a lightweight hand against the disadvantage of complex control, this paper develops an electrical prosthetic hand with a tendon-driven system.It arranges actuators on the outside of the hand and employs wires and tubes as transmitters, as the greater part of the load of current prosthetic hand is an actuator (motor) arranged into the hand.It reduces the load on the remaining limb of the amputee by moving the center of balance away from the hand into the forearm.Moreover, enhancing grasping power, the design includes a proximal joint assisting mechanism in which distal actuators also provide force to the proximal joint.
These mechanisms make the system complex, timedelayed, and nonlinear.In the case that desired trajectory is given, feedback control, represented by Bang-Bang control, and canonical PID control (Ishikawa et al. 2000) has been suggested as the motor control model to realize target-reaching behavior, and in turn employed to control the manipulators.However, when the control object contains components that would cause nonlinear and/or timedelayed responses, these kinds of control would cause large overshoot or oscillation phenomenon.

MAIN CONTROLLER FOR THE EMG PROSTHETIC HAND
The requirements for the controller of an EMG prosthetic hand are summarized as follows.
1.The internal state and system parameters of the controller should be able to change.2. The functions of motion of the EMG sensor-based prosthetic hand should be able to improve.3. The amputee can observe the feedback of the EMG signal pattern.4. The learning mechanism should be able to work even if the evaluation is weak.5.The learning mechanism should be able to run on real time.
The proposed controller based on the On-Line Learning Method (Nishikawa et al. 2000) consists of three units as shown in Figure 3.The units are Analysis Unit, Classification Unit, and Supervision Unit.

Analysis unit
This unit extracts the feature vector V from the EMG signal S. The raw EMG signal is processed using an FFT algorithm.The feature vector is acquired by sampling the frequency spectrum resulting from the FFT for each sensor.We used eight samples for each channel.

Classification unit
This unit classifies the predicted forearm motion from the feature vector V of the Analysis Unit, and also produces the control command θ for the prosthetic hand mechanism.
The system parameters of this unit are learned by using an evaluation from the Supervision Unit.For each motion, the classification receives a package of 16 vectors, which are added to the vector database that is used to calculate the weights of the neural network.The weights are calculated using a back propagation algorithm.

Supervision unit
This unit produces evaluation by using an amputee's instruction θ teach and the feature vectors ς, and these evaluations are sent to the Classification Unit.The procedures of this controller are as follows.The Analysis Unit detects the EMG signal from the sensor on the forearm of the amputee, producing the feature vectors for the Classification Unit.The Classification Unit receives the feature vectors, generating the control commands for the prosthetic hand.If the motion of the prosthetic hand is not equal as the expected motion from the amputee, the operator can include a new data set of training for the Supervision Unit.The Supervision Unit evaluates and updates the Classification Unit until the expected motion is realized.The Classification Unit changes its own system parameters to find the mapping function that is able to denote the relationship of the feature vectors and the expected motion command.Therefore, this controller will be able to produce the suitable mapping function for the individual characteristics of amputee between the EMG signals and the motor command of prosthetic hand.

Experimental result
We carried out the experiments to classify up to 10 forearm motions from two channels of surface EMG using the classifier.We practiced the experiment classifying eight forearm motions by five normal subjects.We executed the experiment classifying 10 forearm motions by three normal subjects who showed high performance in the previous experiments.The detected EMG signal from dry type sensors is amplified 10,000 times to a voltage of near ±10.The amplified signal is digitized using an Analog-Digital acquisition board with a resolution of 12 bits and a sampling rate of 1,600 Hz.The digitized signal is sent to the controller.Dry-type electrodes are used and placed in the neighborhood of the elbow near the origin of forearm muscles, where a number of several different motions can be acquired.Moreover, we can use the same sensor position to amputee subjects with different remaining forearm length.Figure 4 shows the position of the surface electrodes; channel 1 is located at the side of the radius, channel 2 is located on the side of ulna, and channel 3 is used for the detection of the thumb movement.The ground reference is placed on the elbow.
Figure 5 shows the experimental setup.The EMG classifier is implemented by software in a personal computer.We use the keyboard to send the teaching signal to the Supervision Unit.The visual feedback is provided by a computer graphic-generated anthropomorphic hand.This system is used to train the prosthetic system, while providing information on the internal states by the status of the graphic interface.The subject can monitor the performance of the hand, and send a teaching signal, to correct the motion when he or she considers that it is not moving satisfactorily.
We carry out the ability test to measure the efficiency of the classifier to distinguish the different motions.The test subject controls the prosthetic hand according to the instructions presented on the monitor (Fig. 5).To test the motion, we asked the test subject to execute one motion for 3 s.During this time, we calculated the classification error by comparing the current instruction and the output of the classifier.Before starting with the motion testing, there is a training session in which the system is calibrated for the test subject.The test begins when the test subject considers that he or she is able to control the graphic hand.The system can classify up to 10 different forearm motions, which contains four wrist motions and six hand motions.Each motion is assigned an ID number.Figure 6 shows all of the motions and their corresponding ID numbers.The wrist motions are supination (ID:01), pronation (ID:02), flexion (ID:03), and extension (ID:04).The hand motions are 2-5th fingers' flexion (ID:05), 2-5th fingers' extension (ID:06), thumb flexion (ID:07), thumb extension (ID:08), 4-5th fingers' flexion (ID:09), and 2-3rd fingers' extension (ID:10).

ADAPTIVE JOINT MECHANISM
Figure 7 shows a schematic diagram of the adaptive joint mechanism developed at our laboratory (Ishikawa et al. 2000).A spring connects a frame with a wire guide that can shift proportionally to the load applied.In the case of light load as shown in Figure 8(b), the wire approaches the fulcrum, making its angular velocity high and its torque low.On the other side, in the case of heavy load as shown in Figure 8(c), the guide leaves the fulcrum, making its angular velocity low and its torque high.Accordingly, the spring connected to the guide provides the adjustable power transmitting function.The adaptive joint provides a 'passive adaptive grasp' (Hirose and Ma 1991).Dechev et al. (2001) indicate that current prosthetic hands, which are pinch-based devices with rigid fingers, require a high pinch force to secure objects.This gripping force can be reduced using a more flexible mechanism that can adapt to different objects; hence, more flexible hands are needed.Our proposed mechanism achieves this task.
Figure 9 shows the mathematical model of the adaptive joint mechanism.Torque on one articulation (finger tip) (τ ) is derived from the force pulling the wire (F) and an angle with a fulcrum-action line and an action-lever line (θ 1 ), and a distance from the fulcrum to a point of action (L): Here, the angle θ 1 is defined as an angle θ 2 with the fulcrum-action line and a fulcrum-pulling force line, an angle β with the fulcrum-pulley line and the pulling force line, and a distance x from the fulcrum to a point of lever (pulley): (2) If the spring would connect with a point near to the fulcrum, β could approximate a constant value.The distance x is given by the following equation: Here, the parameter k is a Young's modulus of a spring in the direction of fulcrum-guide (pulley) line that When the load is increased (c), the wire moves away from the fulcrum, increasing the torque and reducing the motion speed (Ishikawa et al. 2000).
connects the frame and the guide.These equations can settle the torque τ from the force F, the distance x, and the angle θ 2 .From previous simulation experiments the torque-angular-velocity ratio was 1:6, this was later confirmed with a mechanical prototype (Ishikawa et al. 2000).Furthermore, this mechanism has another function other than the adjustable function of velocity and torque.As shown in Figure 8(a), the passive motion function can be obtained by setting the wire exactly on the center of joint's rotation.

Experiment 1
The following two cases are measured for comparing our proposed mechanism with the conventional mechanism, in which the guide is fixed on x = 1, 5, 9 mm. 1.The angle of the joint, when the wire is pulled with constant speed without load.2. The torque of the joint, when the wire is pulled with constant power and the angle is fixed on 45 • .

Experiment 2
We measured the angle of the joint and the torque of the proposed mechanism under the condition that an obstacle is placed in a workspace so that the finger contacts it when the joint angle is 45 • .Figure 10(a) and (b) show the results of experiments 1-(1) and 1-(2).When the load is light, our proposed mechanism moves faster than the conventional mechanism, in which the guide is fixed on 5 and 9 mm from the fulcrum.On the other hand, when the load is heavy, the finger of our proposed mechanism generated a torque larger than the conventional mechanism with the wire guide fixed on 1 and 5 mm from the fulcrum.Figure 11(a) shows the results of experiment 2. The finger moves fast until it collided to the object.It generated large torque by increasing distance between the guide and the fulcrum, after it collides with the object.Figure 11(b) shows the trajectory of the finger when a free object is placed in the workspace.We can see that the finger moves slowly after it contacts to it, generating larger torque.These simulation results show that desired effect is obtained, using our proposed mechanism.
The mechanism developed by Ishikawa et al. (2000) at our laboratory displayed the compliance with the torquevelocity requirements.Still this mechanism presents several drawbacks.But the application of linear motors as actuators and the spring inclusion in the mechanism make its production and maintenance difficult.The robot hand manages to move the actuator's weight from the hand, using linear motors.The linear motors are difficult to control and are slow.Also the spring mechanism makes the design more expensive.To solve these problems, we proposed the use of spring coil wire as guide for the actuator wire and the use of servomotors as actuators (Fig. 12).
Figure 12(a) and (b) shows the configuration of the mechanical parts for the new implementation.To obtain the spring function of the wire guide in a simple way, we used a spring coil type outer wire as an elastic guide of the inner wire.If the finger does not need a big torque, the springcoil type outer wire keeps straight.If the finger needs a   big torque, the spring-coil type outer wire bends to keep the inner wire taut.The outer wire is made of stainless steel, and boat fishing wire for the inner wire, due to its low friction with the stainless outer wire, and its hightension resistance (37 kg).The new mechanism (Fig. 13) presents the same characteristics to that of the previous mechanism.When the load is light, the outer wire remains straight, keeping the tendon wire close to the fulcrum, resulting in low-torque-high-speed motion (Fig. 11).When the load increases, the outer wire bends, moving the tendon wire away from the fulcrum, increasing the torque, and reducing the actuation speed.igure 15 Statistics for the adaptive joint mechanism for the torque-actuator force relationship when substituting the spring for the spring coil wire.

EMG prosthetic hand
The proposed EMG prosthetic hand has five fingers and wrist.Each finger has three joints; however, the distal-interphalangial (DIP) joint and the proximalinterphalangial (PIP) joint are actuated by a common ten-don wire.The MP joint is actuated by one motor.Thus each finger has 2 DOF with active motion control.Only the thumb carpo-metacarpal joint has been directly connected by servomotor to realize the abduction of motion.The wrist is supported by two motors for the actuation of   The experimental results show the performance of the originally proposed finger.Figure 14(a) shows the relation between the displacement of wire and the force of actuator.The angle (0 • , 45 • , and 90 • ) is measured between finger element and wire direction.This result shows that a bigger angle requires longer wire movement.Figure 14(b) shows the relation between the joint torque and force of actuator.This result shows that a bigger angle produces bigger torque of joint.If the maximum torque of RC servomotor is 3.6 kg cm at 6 V, the maximum torque of finger joint becomes 1.1 kg cm.If there is no obstacle of finger motion, only 5-mm wire movement is enough to rotate the finger joint at 90 • .The maximum velocity of rotation of finger joint became 200 degrees per second by using RC servomotor.The maximum frequency obtained for the taping motion (0 • -90 • ) of each finger is 1 Hz.The application of the spring coil wire as guide showed an increase in the power transference.The new prototype was tested increasing the force pulling the actuator wire, measuring the torque generated at 0 • , 45 • , 90 • of the joint movement.The results show a nonlinear relationship between the force pulling the wire and the force generated.The mechanism showed an increase in the torque available at 90 • (Fig. 15).
Figures 16 and 17 show the prosthetic hand prototype.Figure 18 shows some postures of handling objects: (a) shows the grasping posture of cylindrical form, (b) is the Elliptical case, and (c) is a plastic bottle with juice (1000 mL).The proposed hand can hold 1,000-cc juice in the plastic bottle in the stable state.Figure 17(d) shows the posture to hold a coffee cup; the proposed hand can hold such cup by using only two fingers.Figure 17(e) shows the posture to catch a pen to write character 'A'. Figure 16(e) shows the posture to pinch the CD case by finger tips.The prosthetic hand showed a grip force equal to 400 Nm.

SUMMARY
The prosthetic system based on EMG signal has a big potential to reflect the human intention for the control of a large DOF mechanical device.This paper proposed the adaptive joint mechanism and the EMG-based controller.The experimental results show that the proposed mechanism realizes powerful grasping (400 Nm), which is enough to hold a plastic bottle with 400-cc juice.The proposed prosthetic hand shows high compliance to hold efficiently different objects, from a coffee cup to pinchhold a CD case.The postures of fingers can obtain stable grip and also parallel sliding motion between thumb and other fingers.The controller performed the 10 different patterns of finger motion based on EMG signal patterns.The total weight obtained 1.2 kg, including a controller and batteries, but most of the weight is located away from the hand mechanism, allowing for easier use of the prosthetic hand.The proposed controller opens new possibilities for the smooth manipulation of machines without gears.In our future work, we will study with more detail the performance of the proposed controller and also measure its efficiency during the daily life activities of the amputee.

Figure 4
Figure 4 Surface EMG sensors position.

Figure 5
Figure5Experimental setup.The proposed controller is implemented in a PC.A subject pushes the keyboard to teach and watches a computer graphics hand on the monitor instead of a real-prosthetic hand as visual feedback(Nishikawa et al. 2000).

Figure 6 Figure 7
Figure6It shows the 10 different motions, which can be discriminated from the EMG signals(Nishikawa et al. 2000).

Figure 8
Figure8It shows the principle for the passive joint and adaptive torque mechanism.With light load (b) the wire is close to the fulcrum, resulting in low-torque-high-speed motion.When the load is increased (c), the wire moves away from the fulcrum, increasing the torque and reducing the motion speed(Ishikawa et al. 2000).

Figure 11
Figure 11 (a) Simulation result for experiment 2. (b) Simulation result for experiment 2 when placing an object in the workspace.

Figure 12
Figure 12 Adaptive joint mechanism by using spring coil type outer tube for the tendon guide.(a) The fingers' flexion/extension motion.(b) Power module by using RC servomotor.

Figure 13 .
Figure 13.New adaptive torque mechanism.The spring is substituted for a spring coil outer tube.The spring coil presents similar characteristics to the spring keeping the inner wire (actuation wire) taut at all times.

Figure 16
Figure 16 Aluminum body of finger part and palm part.

Figure 17
Figure 17 Whole products.

Figure 18
Figure 18 From upper left hand to the right: (a) grasping cylindrical form, (b) grasping elliptical form, (c) grasping a plastic bottle, (d) holding a coffee cup, (e) holding a pen, and (f) holding a CD case.