Effective telerehabilitation technologies enable patients with certain physiological disabilities to engage in rehabilitative exercises for performing Activities of Daily Living (ADLs). Therefore, training and assessment scenarios for the performance of ADLs are vital for the promotion for telerehabilitation. In this paper we investigate quantitatively and automatically assessing patient’s kinematic ability to perform functional upper extremity reaching tasks. The shape of the movement trajectory and the instantaneous acceleration of kinematically crucial body parts, such as wrists, are used to compute the approximate entropy of the motions to represent stability (smoothness) in addition to the duration of the activity. Computer simulations were conducted to illustrate the consistency, sensitivity and robustness of the proposed method. A preliminary experiment with kinematic data captured from healthy subjects mimicking a reaching task with dyskinesia showed a high degree of correlation (Cohen’s kappa 0.85 with
In recent decades, with the advancements in telerehabilitation and associated motion capture technologies, an increasing number of research and development activities are focusing on the development of automated quantitative measures of patient performance in Activities of Daily Living (ADL) [
In the past few decades, a number of approaches have been proposed for assessing upper extremities, the majority of which are questionnaires. In musculoskeletal movement disorders of the extremities, most scales are generic. For instance, the self-reported Musculoskeletal Function Assessment (MFA) instrument [
Although these assessment tools have been utilised pervasively by clinicians, they are not suitable for the telerehabilitation environment. One reason is that the use of the majority of these tools requires clinicians, who are usually absent in the telerehabilitation sessions. Furthermore, some self-report questionnaires may lead to biased results. Therefore, in telerehabilitation, it is critical to develop an automated approach to objectively assess the ability of patients in order to assist therapists in making further clinical decisions.
In this paper, we conducted a preliminary investigation of the feasibility of utilising an automatic approach to assess the ability of patients suffering from dyskinesia, to perform an upper extremity reaching task in their daily living. This is assessed by measuring the smoothness of motion trajectories and the duration to finish the task. As is pointed out by Daneault et al. [
The definition of three kinematic severity levels of involuntary movements and jerks, as well as their corresponding abilities in performing reaching tasks in daily living.
Level | 1 | 2 | 3 |
---|---|---|---|
Number of submovements | 0 | ≤3 | >3 |
Amplitude of submovements | 0 |
|
>0.3 m |
Number of jerks | 0 |
|
>3 |
Amplitude of jerks | 0 |
|
>0.03 m |
Duration |
|
5 s~10 s | >10 s |
Ability in performing reaching task | High | Medium | Low |
In line with our work, a number of studies have been conducted to evaluate automated performance measurements or kinematics relating to upper extremities [
Examples of commonly used techniques with features considered for ADL performance measurement. Feature-based performance evaluation (FPE) is primarily based on kinematic or kinetic measurements. Dynamic measurements such as number of velocity peaks [
The contributions of this paper are threefold: Using the shape of the trajectory and instantaneous acceleration to extract kinematic smoothness via the concept of motion entropy. Utilising smoothness and duration as criteria to evaluate the performance of an upper extremity reaching task. Investigating the possibility of using an affordable, noninvasive consumer device for evaluation of an upper extremity reaching task performance evaluation on a regular basis.
In Section
In order to perform computer simulations as well as obtain data from healthy subjects mimicking the underlying involuntary movements, it is important to precisely specify the severity levels of involuntary movements in a kinematic standpoint. Since the frequency and amplitude of involuntary movements in addition to the duration of the specific task are important factors in assessments [
In order to quantitatively evaluate the ability to perform the experimental task, various features need to be extracted from a raw 3D trajectory (
Curvature (
Since the normal approach to compute numerical differentiation is very sensitive to noise, we utilised the approach introduced in [
To compute the approximate entropy of a variable, that is, instantaneous acceleration, we denote it as
A given constant
According to the Frenet-Serret formulas [
The simulations were conducted with Matlab 2013a to ensure that the proposed approach for smoothness measurement met the consistency, sensitivity, and robustness requirements given in [ smoothness measurement should change consistently (either increase or decrease) with respect to the increasing numbers of submovements and jerky movements; smoothness measurement should be sensitive to any variation of smoothness in the trajectory so that more information regarding the smoothness characteristics can be shown; the calculation of smoothness measurement should be robust to measurement noise so that the result of smoothness is less likely to be affected by noise.
To simulate the reaching movement, we utilised the following process.
To ascertain the consistency and sensitivity of the proposed approach, 50 trajectories were generated to simulate a reaching movement with various numbers and amplitudes of involuntary movements and jerks. The specifications for these trajectories are shown in Table
Parameters used to simulate two groups of trajectories. These two groups of trajectories correspond to two severity levels of involuntary movements. The first 25 trajectories belong to the second level with two involuntary movements and two jerks. The last 25 trajectories are in the third level with four involuntary movements and jerks. To simulate the severity in one level, the amplitudes of involuntary movements and jerks increase with the index.
Level 2 | Level 3 | ||
---|---|---|---|
Submovements | Index |
|
|
Duration | 4.5 seconds (135 frames) | 7.5 seconds (210 frames) | |
|
2 | 4 | |
|
45/50 | 40/50 | |
90/50 | 90/60 | ||
— | 140/60 | ||
— | 190/30 | ||
|
Index |
Index |
|
Index |
Index |
||
— | Index |
||
— | Index |
||
|
|||
Jerks |
|
2 | 4 |
|
Index |
Index |
|
Index |
Index |
||
— | Index |
||
— | Index |
To simplify the simulation without losing the effects, we only simulated the trajectories in levels two and three. Furthermore, we assumed the numbers and amplitudes of submovements in three axes of the trajectories are the same, which means
In addition, to evaluate the robustness of different approaches, we generated 100 noiseless trajectories in level two and the numbers and amplitudes of involuntary movements were randomly generated in the given range (refer to Table
For real-data experiment, no film recordings of subjects were made in this study. The Kinect camera provided numerical data that directly related to arm movements. Only deidentified numerical data, representing motion vectors, were stored in the database. Volunteers were researchers and students at Deakin University. Ethics for conducting the experiments in this paper have been approved by Deakin University.
The real-data experiment was conducted with four healthy subjects mimicking three severity levels of involuntary movements (refer to Table
In the experiment, a table, a chair, a book, a second version Kinect, and a laptop were used (refer to Figures
A diagrammatic view of the experimental setup.
Real-data experiment setup image. The top image shows the setup of the Kinect and the subject. The bottom left and right are the RGB and depth images taken from the Kinect. Note that the marker was on the right wrist of the subject (the depth and RGB camera in the Kinect reversed the image).
To accurately track the involuntary movements and jerk, we attached a small infrared reflective marker on the tracking joint (wrist) and made sure the marker always faced the Kinect so that the Kinect could capture the position of the wrist. The data collection program was written with Kinect SDK v2.0-1409 with C# under Windows 8.1. Although there was no precision evaluation on the second version of Kinect, according to [
Four healthy subjects were recruited for the experiment. Their demographic data can be seen in Table
Demographic data of subjects.
Age | Weight (kg) | Height (cm) | Gender | |
---|---|---|---|---|
Subject 1 | 28 | 55 | 172 | Male |
Subject 2 | 29 | 70 | 175 | Male |
Subject 3 | 27 | 60 | 173 | Male |
Subject 4 | 22 | 58 | 160 | Female |
During the experiment, the subject initially held the book with his/her dominant hand (right hand for all subjects) and kept it steady. At the same time, the system operator checked whether the Kinect could capture the marker. If the Kinect could capture the marker and the subject was ready, the system operator gave the subject an instruction to start moving the book and played the music. In the meantime, the Kinect system started recording the position information of the marker into a database for offline analysis. As soon as the subject finished the task (replacing the book in the original position), the system operator stopped the system. Apart from the system operator and the subject, another researcher classified the task (one of the three levels). The manual classification criteria include the duration of finishing the task and numbers and amplitudes of involuntary movements listed in Table
Each subject was required to perform the task at least 30 times in total to ensure that there were at least 10 trials at each level. All the 30 trials were conducted over three days with 10 to 15 trials per day depending on the subject availability. Between each trial, the subject could rest for thirty seconds.
Three simulations were conducted to assess the performance of the proposed approach in terms of motion smoothness, which was evaluated with five approaches, namely, the number of tangential velocity peaks (VP) [
Examples of generated trajectories are depicted in Figure
These three graphs show trajectories in three levels. (a) is in level one for natural movements without involuntary movements and jerks. (b) Trajectory is in the second level with two involuntary movements (with amplitudes of 0.225 and 0.25 meter) and jerks (with amplitudes of 0.025 and 0.0275 meter). (c) is in the third level with four involuntary movements (with amplitudes of 0.45, 0.5, 0.55, and 0.6 meter) and jerks (with amplitude of 0.05, 0.055, 0.06 and 0.065 meter). The red circles are examples of jerks in the second and third level.
The smoothness level of trajectories, which is represented by one over signal-to-noise ratio (SNR) since SNR is nonlinear with respect to the linear change of smoothness, while
Figure
The metric given by these approaches tends to illustrate the consistency characteristics in various approaches used to evaluate the smoothness of trajectories in two severity levels. With the increase of numbers and amplitudes of both involuntary movements and jerks, the smoothness of the trajectories deteriorates. The first half (with blue from 1 to 25) is in the second level and the last half (with red from 26 to 50) is in the third level. The VP was computed with a threshold value of 0.01 meter/s and the temporal gap between two consecutive peaks was 100 ms.
The second aspect was sensitivity (refer to Figure
Sensitivity comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique is preferred to be sensitive to the small changes in the severity of involuntary movements and the change rate metric should be proportional to the change rate of severity. The blue and red lines show the improvement of metrics of the second and third severity levels with respect to the metric of the first trajectory computed with various approaches (refer to (
Lastly, the robustness of the proposed approach was investigated. From Figure
Robustness comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique needs to be limited to a smaller range (on the horizontal axis) in terms of the differences in metrics between the noisy trajectories and their corresponding noiseless ones, and the value with the highest amplitude in vertical axis should be close to zero.
Eventually, ESA outperformed SAL as it was more sensitive to the change of smoothness and also met the requirement of dimensionless, consistency, and robustness.
In the real-data experiment, firstly, the smoothness of all the trajectories was evaluated using the same approaches considered in the computer simulation section. Additionally, by taking the duration into consideration, all these trials were classified into three levels of ability to perform the task by using three commonly used clustering methods, namely,
Here we present the results of our preliminary real-data experiment with healthy subjects mimicking different severity levels of involuntary movements with their upper extremities while seated.
First of all, some examples of simulated trajectories and features are shown in Figure
Examples of trajectories (first three rows), shape models, including curvatures (fourth row) and torsions (fifth row), and instantaneous accelerations (sixth row) are illustrated for three levels of the ability to perform an upper extremity reaching task (columns one to three corresponding to levels one to three of the severity of involuntary movements). The red circles show examples of submovements and green rectangles are examples of jerks.
Secondly, the distributions of various features, including the duration of the task, as well as metrics computed with various approaches, are shown in Figure
The distributions of durations utilised to finish the reaching task, as well as metrics computed by five approaches, for three severity levels. The threshold utilised to compute VP was 0.25 m/s and the temporal gap was 100 ms for all subjects.
Table
Cohen’s kappa (
Approach |
|
GMM | Fuzzy clustering | Average |
---|---|---|---|---|
VP | 0.6875 | 0.6926 | 0.7250 | 0.7017 |
ZCA | 0.7625 | 0.8000 | 0.7500 | 0.7708 |
DJ | 0.5500 | 0.5631 | 0.5500 | 0.5544 |
SAL | 0.7875 | 0.7875 | 0.7500 | 0.7750 |
ESA | 0.8250 | 0.8250 | 0.8500 | 0.8333 |
From the simulation and real-data experiment, the proposed approach has shown its superior performance in terms of capturing nonsmooth movement patterns due to involuntary movements during performance of a specific upper extremity task. The reasons for this are threefold. Firstly, dimensionless and duration-independent entropy was utilised as a metric for evaluation. It is well known that entropy can be utilised to analyse the regularity of variables, which makes it suitable for analysing the ability to perform a task in terms of motion smoothness because motions involving involuntary movements tend to have an irregular trajectory shape. Secondly, apart from considering the dynamics (instantaneous acceleration) of the motion trajectory, the shape of the trajectory was also taken into account, thereby not only meeting the requirement of dimensionless, consistency, and robustness but also more sensitivity than other approaches compared in the computer simulation (refer to Section
However, since this was a preliminary study, there are some areas that require further attention. Firstly, being an affordable device, the Microsoft Kinect is not as accurate as other more expensive commercially available products, such as VICON. Therefore, the lower resolution (especially in
This paper presents a novel approach for the quantitative evaluation of the ability of individuals with involuntary movements to perform reaching tasks involving the upper extremity. We based our approach on the smoothness of the movement trajectory and also the duration of the activity. In particular, the entropy of the shape model and the instantaneous acceleration were used to capture the appropriate performance indices. Experiments with computer simulation and professional role playing mimicking involuntary movements were conducted to provide preliminary validation of the feasibility and performance of using the proposed approach with an affordable Microsoft Kinect. The computer simulation showed the effectiveness of using entropy of the shape and instantaneous acceleration for motion smoothness evaluation in terms of their consistency, sensitivity, and robustness. The real-data experiment results showed that agreement (Cohen’s kappa correlation coefficient) between a human observer and the proposed automated approach with fuzzy clustering in the experiment was 0.8500, compared to 0.7250, 0.7500, 0.5500, and 0.7500 by using the number of tangential velocity peak, number of zero-crossing tangential acceleration, dimensionless jerk, and spectral arc-length, respectively. Further studies involving patients with movement disorders will be conducted in the future to validate the feasibility of the proposed method.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported by the Australian Federal and Victoria State Governments and the Australian Research Council through the ICT Centre of Excellence Program, National ICT Australia (NICTA).