Advances in Human-Computer Interaction

Volume 2013 (2013), Article ID 369425, 9 pages

http://dx.doi.org/10.1155/2013/369425

Review Article

Controlling Assistive Machines in Paralysis Using Brain Waves and Other Biosignals

Paulo Rogério de Almeida Ribeiro,1,2,3 Fabricio Lima Brasil,1,2,4 Matthias Witkowski,1,2 Farid Shiman,1,2 Christian Cipriani,⁵ Nicola Vitiello,⁵ Maria Chiara Carrozza,⁵ and Surjo Raphael Soekadar1,2

¹Institute of Medical Psychology and Behavioral Neurobiology and MEG Center, University of Tübingen, Silcherstraße 5, 72076 Tübingen, Germany

²Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University of Tübingen, Calwerstraße 14, 72076 Tübingen, Germany

³International Max Planck Research School for Neural Information Processing, Österbergstraße 3, 72074 Tübingen, Germany

⁴International Max Planck Research School for Neural & Behavioral Sciences, Österbergstraße 3, 72074 Tübingen, Germany

⁵The BioRobotics Institute, Scuola Superiore Sant’Anna, V.le R. Piaggio 34, 56025 Pontedera, Italy

Received 9 January 2013; Accepted 24 April 2013

Academic Editor: Christoph Braun

Copyright © 2013 Paulo Rogério de Almeida Ribeiro et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The extent to which humans can interact with machines significantly enhanced through inclusion of speech, gestures, and eye movements. However, these communication channels depend on a functional motor system. As many people suffer from severe damage of the motor system resulting in paralysis and inability to communicate, the development of brain-machine interfaces (BMI) that translate electric or metabolic brain activity into control signals of external devices promises to overcome this dependence. People with complete paralysis can learn to use their brain waves to control prosthetic devices or exoskeletons. However, information transfer rates of currently available noninvasive BMI systems are still very limited and do not allow versatile control and interaction with assistive machines. Thus, using brain waves in combination with other biosignals might significantly enhance the ability of people with a compromised motor system to interact with assistive machines. Here, we give an overview of the current state of assistive, noninvasive BMI research and propose to integrate brain waves and other biosignals for improved control and applicability of assistive machines in paralysis. Beside introducing an example of such a system, potential future developments are being discussed.

1. Introduction

The way humans interact with computers has changed substantially in the last decades. While, for many years, the input from the human to the machine was mainly managed through keystrokes, then later through hand movements using a computer mouse, other potential input sources have been opened up allowing more intuitive and effortless control, for example, based on speech [1], gestures [2], or eye movements [3], all depending on a functional motor system.

As cardiovascular diseases increase and people live longer, an increasing number of people suffer from conditions that affect their capacity to communicate or limit their mobility [4], for example, due to stroke, neurodegenerative disorders, or hereditary myopathies. Motor disability can also result from traumatic injuries, affecting the central or peripheral nervous system or can be related to amputations of the upper or lower extremities. While these handicapped people would benefit the most from assistive machines, their capacity to interact with computers or machines is often severely impeded.

Among the most important causes of neurological disabilities resulting in permanent damage and reduction of motor functions or the ability to communicate are stroke, multiple sclerosis (MS), spinal cord injury (SCI), brachial plexus injury (BPI), and neurodegenerative diseases, such as amyotrophic lateral sclerosis (ALS) or dementia [4].

Stroke is the leading cause of long-term disability in adults and affects approximately 20 million people per year worldwide [5, 6]. Five millions remain severely handicapped and dependent on assistance in daily life [4]. Nearly 30% of all stroke patients are under the age of 65 [7]. Other diseases resulting in paralysis at such early age include MS, affecting more than 2.5 million people worldwide [8], or SCI with 12.1 to 57.8 cases per million [9, 10]. BPI, the disruption of the upper limb nerves leading to a flaccid paralysis of the arm, affects thousands of people every year [11]. Furthermore, every year there are approximately 2,000 new traumatic upper limb amputations in Europe [12].

While there is major progress in the development of assistive apparatuses built for instance to compensate for a lost or paralyzed limb for example, lightweight and versatile prostheses or exoskeletons [13–16], intuitive and reliable control of such devices is an enormous challenge.

Previous surveys on the use of artificial hands revealed that up to 50% of the amputees are not using their prosthetic hand regularly, mainly due to low functionality, poor cosmetic appearance, and low controllability [17].

Since early on, the use of electromyographic (EMG) signals for prosthetic control, for example, from the amputee’s stump or contralateral chest muscles, was an important concept [18, 19]. However, its broader success is still limited due to many practical reasons that are valid for all assistive systems that depend on recording biosignals, primarily the effort and costs to provide good signal quality, a fast and effective calibration process, and, last but not least, the benefit of the system in the user’s everyday life. Furthermore, increasing the signal-to-noise ratio or the specificity of such recordings by means of techniques such as the electric nerve stimulation [20] is possible but increases the overall system complexity [21]. Adding sensory qualities during utilization of prosthetic devices increasing the bidirectional interaction between users and the machine improves the functionality of assistive systems [22]. Here, however, the same limitation applies as to the motor domain that the majority of such systems depend on an intact peripheral sensory system.

Thus, the development and provision of assistive machines that are independent of the peripheral nervous system’s integrity represent a promising and appealing perspective, particularly, if controlled intuitively and without requiring extensive training to gain reliable control.

2. Brain-Computer and Brain-Machine Interfaces: A General Overview

Since it was discovered that brain waves contain information about cognitive states [23, 24] and can be functionally specific [25, 26], the idea to use such signals for direct brain control of assistive machines became a major driving force for the development of the so-called brain-computer or brain-machine interfaces (BCI/BMI) [27]. Such interfaces allow direct translation of electric or metabolic brain activity into control signals of external devices or computers bypassing the peripheral nervous and muscular system.

As neural or metabolic brain activity can be recorded from sensors inside or outside the brain, BCI/BMI is categorized as invasive or noninvasive systems [28]. Other categorizations relate to the specific brain signal used for BCI/BMI control or the mode of operation (see Table 1).

Table 1: Categories of brain-computer and brain-machine interfaces.

Based on: recording site of brain signals

Brain signal used
Recording technique

Invasive

    Single spike
Single cell recordings

    Multiunit activity
Multiunit arrays (MUA)

    Local field potentials (LFP)
Electrocorticogram (ECoG)

Noninvasive

    Electric brain potentials
Electroencephalography (EEG)

    Neuromagnetic fields
Magnetoencephalography (MEG)

    BOLD
Functional magnetic resonance imaging (fMRI)

    Oxy/deoxyhemoglobin
Near-infrared spectroscopy (NIRS)

Based on: mode of operation

Active
Asynchronous control

Synchronous control

Reactive
N.A.

Passive
N.A.

Based on: purpose

Assistive/biomimetic
Restorative/biofeedback

Used for restoration of
Tested in the treatment of

    Communication
    Stroke

    Paralysis
    Chronic pain

    Tinnitus

    Dementia

    Depression

    Schizophrenia

Invasively recorded brain signals that were successfully used for BCI/BMI control include single-spike or multiunit activity and local field potentials (LFP) [29]. These signals are necessarily recorded from inside the skull, while electric or magnetic brain oscillations reflecting pattern formation of larger cell assemblies’ activity [30] can also be recorded from outside the skull using electro- or magnetoencephalography (EEG/MEG). Each method offers access to specific unique properties of brain activity [31]. These noninvasive techniques allow, for example, detection and translation of slow cortical potentials (SCP), changes of sensorimotor rhythms (SMR), or event-related potentials (ERP), for example, the P300, translating them into control signals for external devices or computers. More recently, online interpretation of changes in metabolic brain activity [32, 33] was introduced for BCI/BMI application offering high spatial (in the range of mm), but low temporal, resolution (in the range of seconds). These systems use functional magnetic resonance imaging (fMRI) [32] or near-infrared spectroscopy (NIRS) [33, 34], both measuring changes in brain tissue’s blood-oxygenation-level dependent (BOLD) signals.

In 1969, Fetz demonstrated that single neurons in precentral cortex can be operantly conditioned by delivery of food pellets [35]. Since then, operant conditioning of cortical activity was demonstrated in various paradigms [36], requiring, though, opening of the skull and insertion of electrodes into the brain with the risk of bleedings and infections [37, 38]. An intermediate, semiinvasive approach uses LFP recorded by epidural electrocorticography (ECoG) [29, 39]. LFP reflects neural activity of an area of up to 200 μm² comprising hundreds of thousands of neurons with numerous local recurrent connections and connections to more distant brain regions [40], while brain oscillations recorded noninvasively (e.g., using EEG or MEG) contain information of millions of neurons [41].

To control assistive devices or machines in paralysis, the following noninvasively recorded neurophysiologic signals were successfully used up to now: (1) slow cortical potentials (SCP) [42, 43], (2) sensorimotor rhythms (SMRs) and its harmonics [44, 45], and (3) event-related potentials (ERPs), for example, P300 [46].

The use of SCP in BCI/BMI applications goes back to Birbaumer and his coworker’s work in the late 1970s showing that operant control of SCPs (slow direct-current shifts occurring event-related after 300 ms to several seconds) is possible while exhibiting strong and anatomically specific effects on behavior and cognition [47–49]. A tight correlation of central SCPs and blood-oxygen level-dependent (BOLD) signals in the anterior basal ganglia and premotor cortex was found [50] suggesting a critical role of the basal ganglia-thalamo-frontal network for operant control of SCP.

In contrast to SCPs, SMRs are recorded over the sensorimotor cortex usually at a frequency between 8 and 15 Hz. In analogy to the occipital alpha and visual processing [51], the SMR (or rolandic alpha) shows a clear functional specificity, disappearing during planned, actual, or imagined movements [52]. Accordingly, a close association with functional motor inhibition of thalamocortical loops was suggested [53]. Depending on the context, the SMR is also called μ-rhythm [54] or rolandic alpha and was extensively investigated by the Pfurtscheller group in Graz [55] and the Wolpaw group in Albany [56, 57].

Another well-established and tested BCI/BMI controller is the P300-based ERP-BCI introduced by Farwell and Donchin [58]. While SCP- and SMR-controls are learned through visual and auditory feedback often requiring multiple training sessions before reliable control is achieved, the P300-BCI needs no training at all. While, in the classical P300-ERP-BCI paradigm, the user focuses his attention to a visual stimulus, other sensory qualities such as tactile [59] or auditory stimuli [60, 61] were successfully implemented in ERP-BCI. Information rates of ERP-BCI can reach 20–30 bits/min: [62].

In terms of operation mode, active, passive, and reactive BCI/BMI applications can be distinguished [63]. While active and reactive BCI/BMI require the user’s full attention to generate voluntary and directed commands, passive BCI/BMI relates to the concept of cognitive monitoring introducing the assessment of the users’ intentions, situational interpretations, and emotional states [64].

In active BCI/BMI applications, two forms of control can be distinguished: synchronous and asynchronous control [65]. In synchronous control, translation of brain activity follows a fixed sequence or cue. The user is required to be fully attentive, while in asynchronous or uncued control, a specific brain signal is used to detect the user’s intention to engage in BCI/BMI control [65, 66].

3. Brain-Machine Interfaces in Neurorehabilitation of Paralysis

BMI used in neurorehabilitation follows two different strategies: while assistive or biomimetic BMI systems strive for continuous high-dimensional control of robotic devices or functional electric stimulation (FES) of paralyzed muscles to substitute for lost motor functions in a daily life environment [67–69], restorative or biofeedback BMI systems aim at normalizing of neurophysiologic activity that might facilitate motor recovery [70–74]. Insofar, restorative or biofeedback BMI can be considered as “training-tools” to induce use-dependent brain plasticity increasing the patient’s capacity for motor learning [44, 75].

These two approaches derive from different research traditions and are not necessarily related to the invasiveness of the approach: in the early 80s of the last century, decoding of different movement directions from single neurons was successfully demonstrated [76]. Since then, reconstruction of complex movements from neuronal activity was pursued, using both invasive and noninvasive methods.

Firing patterns acquired through single cell recordings from the motor cortex [77] or parietal neuronal pools [78] in animals were remarkably successful for reconstruction of movement trajectories. Monkeys learned to control computer cursors towards moving targets on a screen activating neurons in motor, premotor, and parietal motor areas. It was shown that 32 cells were sufficient to move an artificial arm and perform skillful reaching movements enabling a monkey to feed himself [67]. Learned control of movements based on single cell activity was also shown using neurons outside the primary or secondary motor representations [79]. In 2006, successful implantation of densely packed microelectrode arrays in two quadriplegic human patients was demonstrated, enabling them to use LFP in order to move a computer cursor in several directions [68]. Most recently, a study using two 96-channel intracortical microelectrodes placed in the motor cortex of a 52-year-old woman with tetraplegia demonstrated robust seven-dimensional movements of a prosthetic limb [80].

In contrast to this work aiming at assistive appliance of invasive and noninvasive BMI technology, the development of restorative/biofeedback BMI systems is tightly associated with the development and successes of neurofeedback (NF) and its use to purposefully upregulate or downregulate brain activity—a quality that showed to have some beneficial effect in the treatment of various neurological and psychiatric disorders associated with neurophysiologic abnormalities [71]. In NF, subjects receive visual or auditory online feedback of their brain activity and are asked to voluntarily modify, for example, a particular type of brainwave. Successful modification becomes contingently rewarded. NF was successfully used in the treatment of epilepsy [81, 82], ADHD [83–85], chronic pain syndrome [86]. The rational to use this approach in the context of neurorehabilitation is based on data indicating that stroke patients with best motor recovery are the ones in whom ipsilesional cortical function is closer to that found in healthy controls [87]. A negative correlation between impairment and activation in ipsilesional M1 during hand motions has been documented [88]. Thus, a larger clinical study was performed at the University of Tübingen in Germany and the National Institute of Neurological Disorders and Stroke (NINDS, NIH) in USA with over 30 chronic stroke patients testing the hypothesis that augmentation of ipsilesional brain activity would improve motor recovery [89, 90]. In this study, all participating patients suffered from complete hand paralysis and were unable, for example, to grasp. The study showed that one month of daily ipsilesional BMI training combined with goal-directed physiotherapy resulted in significant motor improvements, while random BMI-feedback did not. Further analysis of neurophysiological parameters indicated that motor evoked potentials (MEP) from the ipsilesional hemisphere reflecting the integrity of the corticospinal tract could predict motor recovery of the trained patients [91]. Currently, further improvements of this training paradigm, for example, related to the feedback or specificity and effectiveness of training [44], for example, using electric brain stimulation to enhance neuroplasticity [92], are being tested.

4. Noninvasive Assistive Brain-Machine Interfaces in Paralysis

Both invasive and noninvasive BCI and BMI found their way into assistive systems, for example, allowing communication in locked-in patients [42] or restoration of movement in patients with paralysis [28, 93]. The Graz group was the first to use volitional SMR modulation for control of electric stimulation of a quadriplegic patient’s paralyzed hand [69, 94]. While the patient imagined a movement, the associated modulation of SMR was translated into functional electric stimulation (FES) of his upper limb muscles resulting in grasping motions. After this proof-of-concept study, numerous publications addressed the different aspects that are important to allow intuitive and seamless control of biomimetic devices [20] or FES [95] in a daily life environment [96]. While many challenges were successfully mastered in the last years, three major aspects were not satisfyingly solved yet: (1) intuitive, asynchronous BCI/BMI control, (2) 100% reliability, and (3) unambiguous superiority (in terms of information transfer rate, ITR, and necessary preparation effort) over the use of other biosignals (e.g., related to speech, gestures, or eye movements).

These aspects do not apply to BCI use for communication in complete paralysis, for example, complete locked-in-state (CLIS) in ALS, as no asynchronous mode is necessary, reliability is secondary, and no other biosignals are available anymore [97].

A system that is unreliable in daily does not only limit its practicality, but limits its practicality, but would be also associated with ethical difficulties [98, 99]. While there are good arguments suggesting that invasive BCI/BMI can provide a higher ITR [100], it is still unclear how much meaningful information, for example, for reconstruction of hand movements, can be extracted from noninvasively recorded brain signals [101]. Recently, work by Contreras-Vidal’s group at the University of Houston suggested that slow-frequency EEG (oscillations with a frequency of up to 4 Hz) might provide as much information as invasive recordings [102, 103], for example, for reconstruction of three-dimensional hand movements [103]. Currently, implementation of this approach in closed-loop paradigms is being pursued. Nevertheless, it is conceivable that the only viable solution to satisfyingly solve those three aspects will be the inclusion of other biosignals into a system merging different biosignal sources to detect user’s intentions and integrating this information into the current context of the user to further increase intuitive control and assure reliability of the system. Such systems that merge brain control with other biosignals were recently summarized under the term “brain-neural computer interaction” (BNCI) systems receiving notable funding through the 7th Framework Program for Research and Technological Development (FP7) of the European Union.

Particularly promising in this context is integrating eye movements using electrooculography (EOG) or eye tracking into prosthetic control. At the University of Tübingen, a first prototype system was conceptualized that allows asynchronous BCI/BMI control while solving the reliability issue by using eye tracking, EOG, and computer vision-based object recognition. A computer equipped with a 3D camera recognizes objects placed on a table. The system detects when the user fixates any of the objects recognized as graspable, for example, a cup or ball. Once an object is fixated with the eyes, the BCI/BMI mode switches on, detecting whether the user wants to grasp the object. A robotic hand or exoskeleton (both developed by the BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy) performs the grasping motion (Figure 1). The motion becomes interrupted if the user does not fixate the object anymore as measured by eye tracking and EOG (see Figure 2). This assures that no action of the system depends exclusively on brain wave control that might be susceptible to inaccuracies. Such system, integrating perceptual and contextual computing developed in the field of human-computer interaction (HCI) research into BCI applications, promises to overcome many limitations of brain control alone, mainly the reliability issue, likewise broadening the repertoire of modern HCI research to infer user state and intention from brain activity.

Figure 1: Organization of the University of Tübingen’ prototype system controlling assistive devices using brain waves and eye movements.

Figure 2: Illustration of the processing chain for performing grasping motions of an assistive system using brain waves and eye movements. The grasping motion stops once the user does not fixate the object with his eyes anymore.

As trauma or stroke can affect motor and body functions very differently in each individual, proper and fast calibration for inclusion into seamless BNCI control is often impeded. Thus, inclusion of eye movements is the most promising biosignal in this context so far. Particularly as visual interaction plays a key role when planning, executing, and adapting motor control. Beside electric biosignals such as EOG and EMG, other measures that can be used for BNCI control include magnetic, mechanic, optic, acoustic, chemical, and thermal biosignals. These biosignals, however, are more susceptible for artifacts and exhibit larger variability depending on the environmental conditions. Future research, however, might find novel ways to advantageously include such biosignals into BNCI control and application.

The organisms’ behavior measurable in these various biosignals reflects conscious and unconscious processes that can be inferred and purposefully used for BNCI control. In case of eye movement control, changing fixation of an object can point to inattention, distraction, or volitional (conscious) act to interrupt unwanted output of the BNCI for example.

Practicality of such approach is limited when, for instance, eyesight or eyeball control is impaired due to a disease or trauma. This can be the case in multiple sclerosis, traumatic brain injury, stroke, or neurodegenerative disorders such as ALS. ALS may lead to CLIS, where classical semantic conditioning might be the only way to sustain a communication channel [104] while inclusion or use of other biosignals seemed not particularly helpful [94]. Also, inclusion of other biosignals often increases preparation time for placing and calibrating the required biosensors further limiting practicality. This is particularly relevant when the system requires handicapped persons to place and handle the sensors in a home environment. Nevertheless, these technical limitations might dissolve in the course of near-future research and development.

An important conceptual advantage of including other biosignals into BCI control relates to the improved reliability, which not only increases usability in daily life, but also the degree of self-efficacy, a dimension that should not be underestimated in acceptance of such technology, but also in the context of restorative/biofeedback BCI training for example. Here, the fact that a patient experiences full control of a completely paralyzed limb might facilitate overcoming “learned nonuse” and motivate the user to engage in behavioral physiotherapy [105].

5. Conclusion

BCI/BMI systems promise to enhance applicability of assistive technology in humans with a compromised or damaged motor system. While information transfer rates of noninvasive BCI/BMI are sufficient for communication, for example, in locked-in-state, versatile control of prosthetic devices using brain waves will require major research and development efforts to provide intuitive, asynchronous control sufficiently reliable in daily life environments. Many reasons suggest that using the combination of brain waves with other biosignals might entail many attractive solutions to control assistive, noninvasive technology even after severe damage of the central or peripheral nervous system.

Authors’ Contribution

Paulo Rogério de Almeida Ribeiro and Fabricio Lima Brasil contributed equally to this work.

Acknowledgments

This work was supported by the EU Project WAY FP7-ICT-2011-288551, the Italian Project AMULOS (Industria 2015, MI01 00319), the Regional Project EARLYREHAB (Regione Toscana, Health Regional Research Programme 2009), the German Federal Ministry of Education and Research (BMBF, 01GQ0831, and 16SV5840), and the Deutsche Forschungsgemeinschaft (DFG SO932-2), Open Access Publishing Fund of the University of Tübingen, as well as CNPq/DAAD (National Council for Scientific and Technological Development—Brazil; German Academic Exchange Service—Germany) scholarships.

References

S. Furui, “50 years of progress in speech and speaker recognition research,” ECTI Transactions on Computer and Information Technology, vol. 1, no. 2, p. 64, 2005.

H. S. Yoon, J. Soh, Y. J. Bae, and H. Seung Yang, “Hand gesture recognition using combined features of location, angle and velocity,” Pattern Recognition, vol. 34, no. 7, pp. 1491–1501, 2001.

M. R. Ahsan, M. I. Ibrahimy, and O. O. Khalifa, “EMG signal classification for human computer interaction: a review,” European Journal of Scientific Research, vol. 33, no. 3, pp. 480–501, 2009.

W. H. O., World report on disability: World Health Organization, 2011

N. S. Ward and L. G. Cohen, “Mechanisms underlying recovery of motor function after stroke,” Archives of Neurology, vol. 61, no. 12, pp. 1844–1848, 2004.

S. MacMahon, “Introduction: the global burden of stroke,” in Clinician’s Manual on Blood Pressure and Stroke Prevention, J. Chalmers, Ed., pp. 1–6, Science Press, London, UK, 2002.

N. I. O. N. Disorders and Stroke, Stroke: hope through research, National Institute of Neurological Disorders and Stroke, National Institutes of Health, 1999

T. C. Frohman, D. L. O'Donoghue, and D. Northrop, A Practical Primer: Multiple Sclerosis for the Physician Assistant, Southwestern Medical Center, Dallas, Tex, USA, 2011.

F. W. A. Van Asbeck, M. W. M. Post, and R. F. Pangalila, “An epidemiological description of spinal cord injuries in The Netherlands in 1994,” Spinal Cord, vol. 38, no. 7, pp. 420–424, 2000.

F. Martins, F. Freitas, L. Martins, J. F. Dartigues, and M. Barat, “Spinal cord injuries—epidemiology in Portugal's central region,” Spinal Cord, vol. 36, no. 8, pp. 574–578, 1998.

W. Pondaag, M. J. A. Malessy, J. G. Van Dijk, and R. T. W. M. Thomeer, “Natural history of obstetric brachial plexus palsy: a systematic review,” Developmental Medicine & Child Neurology, vol. 46, no. 2, pp. 138–144, 2004.

S. Banzi, E. Mainardi, and A. Davalli, “Analisi delle strategie di controllo per protesi di arto superior in pazienti con amputazioni transomerali o disarticolati di spalla,” in Biosys, ANIPLA, pp. 290–300, 2005.

J. L. Pons, “Rehabilitation exoskeletal robotics. The promise of an emerging field,” IEEE Engineering in Medicine and Biology Magazine, vol. 29, no. 3, pp. 57–63, 2010.

N. Vitiello, T. Lenzo, S. Roccella et al., “NEUROExos: a powered elbow exoskeleton for physical rehabilitation,” IEEE Transactions on Robotics, vol. 29, no. 1, pp. 220–235, 2013.

A. Chiri, N. Vitiello, F. Giovacchini, S. Roccella, F. Vecchi, and M. C. Carrozza, “Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation,” IEEE/ASME Transactions on Mechatronics, vol. 17, no. 5, pp. 884–894, 2012.

J. Iqbal, N. G. Tsagarakis, A. E. Fiorilla, and D. G. Caldwell, “A portable rehabilitation device for the hand,” in Proceedings of the 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '10), pp. 3694–3697, September 2010.

D. J. Atkins, D. C. Y. Heard, and W. H. Donovan, “Epidemiologic overview of individuals with upper-limb loss and their reported research priorities,” Journal of Prosthetics and Orthotics, vol. 8, no. 1, pp. 2–11, 1996.

B. Peerdeman, D. Boere, H. Witteveen, et al., “Myoelectric forearm prostheses: state of the art from a user-centered perspective,” Journal of Rehabilitation Research and Development, vol. 48, no. 6, pp. 719–737, 2011.

M. Zecca, S. Micera, M. C. Carrozza, and P. Dario, “Control of multifunctional prosthetic hands by processing the electromyographic signal,” Critical Reviews in Biomedical Engineering, vol. 30, no. 4-6, pp. 459–485, 2002.

R. Rupp and H. J. Gerner, “Neuroprosthetics of the upper extremity—clinical application in spinal cord injury and challenges for the future,” in Operative Neuromodulation, vol. 97 of Acta Neurochirurgica Supplements, pp. 419–426, 2007.

A. Fougner, O. Stavdahl, P. J. Kyberd, Y. G. Losier, and P. A. Parker, “Control of upper limb prostheses: terminology and proportional myoelectric control—a review,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 20, no. 5, pp. 663–677, 2012.

D. J. Weber, R. Friesen, and L. E. Miller, “Interfacing the somatosensory system to restore touch and proprioception: essential considerations,” Journal of Motor Behavior, vol. 44, no. 6, pp. 403–418, 2012.

W. J. Ray and H. W. Cole, “EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes,” Science, vol. 228, no. 4700, pp. 750–752, 1985.

W. Klimesch, “EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis,” Brain Research Reviews, vol. 29, no. 2-3, pp. 169–195, 1999.

G. E. Chatrian, M. C. Petersen, and J. A. Lazarte, “The blocking of the rolandic wicket rhythm and some central changes related to movement,” Electroencephalography and Clinical Neurophysiology, vol. 11, no. 3, pp. 497–510, 1959.

E. G. Walsh, “‘Visual attention’ and the alpha-rhythm,” The Journal of Physiology, vol. 120, no. 1-2, pp. 155–159, 1953.

J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767–791, 2002.

N. Birbaumer, “Breaking the silence: brain-computer interfaces (BCI) for communication and motor control,” Psychophysiology, vol. 43, no. 6, pp. 517–532, 2006.

G. Schalk and E. C. Leuthardt, “Brain-computer interfaces using electrocorticographic signals,” IEEE Reviews in Biomedical Engineering, vol. 4, pp. 140–154, 2011.

F. Lopes da Silva, “Neural mechanisms underlying brain waves: from neural membranes to networks,” Electroencephalography and Clinical Neurophysiology, vol. 79, no. 2, pp. 81–93, 1991.

J. Malmivuo, “Comparison of the properties of EEG and MEG in detecting the electric activity of the brain,” Brain Topography, vol. 25, no. 1, pp. 1–19, 2012.

N. Weiskopf, R. Veit, M. Erb et al., “Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data,” NeuroImage, vol. 19, no. 3, pp. 577–586, 2003.

R. Sitaram, H. Zhang, C. Guan et al., “Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface,” NeuroImage, vol. 34, no. 4, pp. 1416–1427, 2007.

T. Nagaoka, K. Sakatani, T. Awano et al., “Development of a new rehabilitation system based on a brain-computer interface using near-infrared spectroscopy,” in Oxygen Transport to Tissue XXXI, vol. 662 of Advances in Experimental Medicine and Biology, pp. 497–503, 2010.

E. E. Fetz, “Operant conditioning of cortical unit activity,” Science, vol. 163, no. 3870, pp. 955–958, 1969.

E. E. Fetz, “Volitional control of neural activity: implications for brain-computer interfaces,” The Journal of Physiology, vol. 579, no. 3, pp. 571–579, 2007.

E. Behrens, J. Schramm, J. Zentner, and R. König, “Surgical and neurological complications in a series of 708 epilepsy surgery procedures,” Neurosurgery, vol. 41, no. 1, pp. 1–10, 1997.

A. M. Korinek, J. L. Golmard, A. Elcheick et al., “Risk factors for neurosurgical site infections after craniotomy: a critical reappraisal of antibiotic prophylaxis on 4578 patients,” British Journal of Neurosurgery, vol. 19, no. 2, pp. 155–162, 2005.

D. Moran, “Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms,” Current Opinion in Neurobiology, vol. 20, no. 6, pp. 741–745, 2010.

J. Linke, S. H. Witt, A. V. King et al., “Genome-wide supported risk variant for bipolar disorder alters anatomical connectivity in the human brain,” NeuroImage, vol. 59, no. 4, pp. 3288–3296, 2012.

D. Turner, P. Patil, and M. Nicolelis, “Conceptual and technical approaches to human neural ensemble recordings,” in Methods for Neural Ensemble Recordings, M. A. L. Nicolelis, Ed., Boca Raton, Fla, USA, 2nd edition, 2008.

N. Birbaumer, N. Ghanayim, T. Hinterberger et al., “A spelling device for the paralysed,” Nature, vol. 398, no. 6725, pp. 297–298, 1999.

T. Elbert, B. Rockstroh, W. Lutzenberger, and N. Birbaumer, “Biofeedback of slow cortical potentials. I,” Electroencephalography and Clinical Neurophysiology, vol. 48, no. 3, pp. 293–301, 1980.

S. R. Soekadar, M. Witkowski, J. Mellinger, A. Ramos, N. Birbaumer, and L. G. Cohen, “ERD-based online brain-machine interfaces (BMI) in the context of neurorehabilitation: optimizing BMI learning and performance,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 19, no. 5, pp. 542–549, 2011.

C. Neuper, G. R. Müller-Putz, R. Scherer, and G. Pfurtscheller, “Motor imagery and EEG-based control of spelling devices and neuroprostheses,” Progress in Brain Research, vol. 159, pp. 393–409, 2006.

G. R. Müller-Putz and G. Pfurtscheller, “Control of an electrical prosthesis with an SSVEP-based BCI,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 1, pp. 361–364, 2008.

N. Birbaumer, T. Elbert, A. G. M. Canavan, and B. Rockstroh, “Slow potentials of the cerebral cortex and behavior,” Physiological Reviews, vol. 70, no. 1, pp. 1–41, 1990.

N. Birbaumer, L. E. Roberts, W. Lutzenberger, B. Rockstroh, and T. Elbert, “Area-specific self-regulation of slow cortical potentials on the sagittal midline and its effects on behavior,” Electroencephalography and Clinical Neurophysiology, vol. 84, no. 4, pp. 353–361, 1992.

N. Birbaumer, H. Flor, W. Lutzenberger, and T. Elbert, “Chaos and order in the human brain,” Electroencephalography and Clinical Neurophysiology, vol. 44, pp. 450–459, 1995.

T. Hinterberger, R. Veit, B. Wilhelm, N. Weiskopf, J. J. Vatine, and N. Birbaumer, “Neuronal mechanisms underlying control of a brain—computer interface,” European Journal of Neuroscience, vol. 21, no. 11, pp. 3169–3181, 2005.

P. Eberlin and D. Yager, “Alpha blocking during visual after-images,” Electroencephalography and Clinical Neurophysiology, vol. 25, no. 1, pp. 23–28, 1968.

R. C. Howe and M. B. Sterman, “Cortical-subcortical EEG correlates of suppressed motor behavior during sleep and waking in the cat,” Electroencephalography and Clinical Neurophysiology, vol. 32, no. 6, pp. 681–695, 1972.

C. Neuper and G. Pfurtscheller, “Event-related dynamics of cortical rhythms: frequency-specific features and functional correlates,” International Journal of Psychophysiology, vol. 43, no. 1, pp. 41–58, 2001.

H. Gastaut, “Electrocorticographic study of the reactivity of rolandic rhythm,” Revue Neurologique, vol. 87, no. 2, pp. 176–182, 1952.

G. Pfurtscheller, C. Neuper, and N. Birbaumer, “Human brain-computer interface (BCI),” in Motor Cortex in Voluntary Movements. A Distributed System for Distributed Functions, pp. 367–401, 2005.

J. R. Wolpaw and D. J. McFarland, “Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans,” Proceedings of the National Academy of Sciences of the United States of America, vol. 101, no. 51, pp. 17849–17854, 2004.

J. R. Wolpaw, “Brain-computer interfaces as new brain output pathways,” The Journal of Physiology, vol. 579, no. 3, pp. 613–619, 2007.

L. A. Farwell and E. Donchin, “Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalography and Clinical Neurophysiology, vol. 70, no. 6, pp. 510–523, 1988.

A. Chatterjee, V. Aggarwal, A. Ramos, S. Acharya, and N. V. Thakor, “A brain-computer interface with vibrotactile biofeedback for haptic information,” Journal of NeuroEngineering and Rehabilitation, vol. 4, article 40, 2007.

I. Käthner, C. A. Ruf, E. Pasqualotto, C. Braun, N. Birbaumer, and S. Halder, “A portable auditory P300 brain-computer interface with directional cues,” Clinical Neurophysiology, vol. 124, no. 2, pp. 327–338, 2012.

M. Schreuder, B. Blankertz, and M. Tangermann, “A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue,” PLoS ONE, vol. 5, no. 4, Article ID e9813, 2010.

A. Lenhardt, M. Kaper, and H. J. Ritter, “An adaptive P300-based online brain-computer interface,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 16, no. 2, pp. 121–130, 2008.

T. O. Zander and C. Kothe, “Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general,” Journal of Neural Engineering, vol. 8, no. 2, Article ID 025005, 2011.

T. Zander and S. Jatzev, “Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment,” Journal of Neural Engineering, vol. 9, no. 1, Article ID 016003, 2012.

G. R. Müller-Putz, R. Scherer, G. Pfurtscheller, and R. Rupp, “Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation,” Biomedizinische Technik, vol. 51, no. 2, pp. 57–63, 2006.

S. G. Mason and G. E. Birch, “A brain-controlled switch for asynchronous control applications,” IEEE Transactions on Biomedical Engineering, vol. 47, no. 10, pp. 1297–1307, 2000.

M. Velliste, S. Perel, M. C. Spalding, A. S. Whitford, and A. B. Schwartz, “Cortical control of a prosthetic arm for self-feeding,” Nature, vol. 453, no. 7198, pp. 1098–1101, 2008.

L. R. Hochberg, M. D. Serruya, G. M. Friehs et al., “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, vol. 442, no. 7099, pp. 164–171, 2006.

G. Pfurtscheller, C. Guger, G. Müller, G. Krausz, and C. Neuper, “Brain oscillations control hand orthosis in a tetraplegic,” Neuroscience Letters, vol. 292, no. 3, pp. 211–214, 2000.

N. Birbaumer and L. G. Cohen, “Brain-computer interfaces: communication and restoration of movement in paralysis,” The Journal of Physiology, vol. 579, no. 3, pp. 621–636, 2007.

N. Birbaumer, A. Ramos Murguialday, C. Weber, and P. Montoya, “Neurofeedback and brain-computer interface: clinical applications,” International Review of Neurobiology, vol. 86, pp. 107–117, 2009.

J. J. Daly and J. R. Wolpaw, “Brain-computer interfaces in neurological rehabilitation,” The Lancet Neurology, vol. 7, no. 11, pp. 1032–1043, 2008.

D. Broetz, C. Braun, C. Weber, S. R. Soekadar, A. Caria, and N. Birbaumer, “Combination of brain-computer interface training and goal-directed physical therapy in chronic stroke: a case report,” Neurorehabilitation and Neural Repair, vol. 24, no. 7, pp. 674–679, 2010.

A. Caria, C. Weber, D. Brötz et al., “Chronic stroke recovery after combined BCI training and physiotherapy: a case report,” Psychophysiology, vol. 48, no. 4, pp. 578–582, 2010.

W. Wang, J. L. Collinger, M. A. Perez et al., “Neural interface technology for rehabilitation: exploiting and promoting neuroplasticity,” Physical Medicine and Rehabilitation Clinics of North America, vol. 21, no. 1, pp. 157–178, 2010.

A. P. Georgopoulos, A. B. Schwartz, and R. E. Kettner, “Neuronal population coding on movement direction,” Science, vol. 233, no. 4771, pp. 1416–1419, 1986.

M. A. L. Nicolelis, D. Dimitrov, J. M. Carmena et al., “Chronic, multisite, multielectrode recordings in macaque monkeys,” Proceedings of the National Academy of Sciences of the United States of America, vol. 100, no. 19, pp. 11041–11046, 2003.

H. Scherberger, M. R. Jarvis, and R. A. Andersen, “Cortical local field potential encodes movement intentions in the posterior parietal cortex,” Neuron, vol. 46, no. 2, pp. 347–354, 2005.

D. M. Taylor, S. I. H. Tillery, and A. B. Schwartz, “Direct cortical control of 3D neuroprosthetic devices,” Science, vol. 296, no. 5574, pp. 1829–1832, 2002.

M. Velliste, A. McMorland, E. Diril, S. Clanton, and A. Schwartz, “State-space control of prosthetic hand shape,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '12), pp. 964–967, 2012.

A. R. Seifert and J. F. Lubar, “Reduction of epileptic seizures through EEG biofeedback training,” Biological Psychology, vol. 3, no. 3, pp. 157–184, 1975.

B. Kotchoubey, U. Strehl, C. Uhlmann et al., “Modification of slow cortical potentials in patients with refractory epilepsy: a controlled outcome study,” Epilepsia, vol. 42, no. 3, pp. 406–416, 2001.

N. Birbaumer, T. Elbert, B. Rockstroh, and W. Lutzenberger, “Biofeedback of slow cortical potentials in attentional disorders,” in Cerebral Psychophysiology: Studies in Event-Related Potentials, pp. 440–442, 1986.

U. Strehl, U. Leins, G. Goth, C. Klinger, T. Hinterberger, and N. Birbaumer, “Self-regulation of slow cortical potentials: a new treatment for children with attention-deficit/hyperactivity disorder,” Pediatrics, vol. 118, no. 5, pp. e1530–e1540, 2006.

T. Fuchs, N. Birbaumer, W. Lutzenberger, J. H. Gruzelier, and J. Kaiser, “Neurofeedback treatment for attention-deficit/hyperactivity disorder in children: a comparison with methylphenidate,” Applied Psychophysiology and Biofeedback, vol. 28, no. 1, pp. 1–12, 2003.

M. Lotze, W. Grodd, N. Birbaumer, M. Erb, E. Huse, and H. Flor, “Does use of a myoelectric prosthesis prevent cortical reorganization and phantom limb pain?” Nature Neuroscience, vol. 2, no. 6, pp. 501–502, 1999.

T. Platz, I. H. Kim, U. Engel, A. Kieselbach, and K. H. Mauritz, “Brain activation pattern as assessed with multi-modal EEG analysis predict motor recovery among stroke patients with mild arm paresis who receive the Arm Ability Training,” Restorative Neurology and Neuroscience, vol. 20, no. 1-2, pp. 21–35, 2002.

C. Calautti, M. Naccarato, P. S. Jones et al., “The relationship between motor deficit and hemisphere activation balance after stroke: a 3T fMRI study,” NeuroImage, vol. 34, no. 1, pp. 322–331, 2007.

E. Buch, C. Weber, L. G. Cohen et al., “Think to move: a neuromagnetic brain-computer interface (BCI) system for chronic stroke,” Stroke, vol. 39, no. 3, pp. 910–917, 2008.

A. Ramos-Murguialday, D. Broetz, M. Rea, et al., “Brain-machine-interface in chronic stroke rehabilitation: a controlled study,” Annals of Neurology, 2013.

F. Brasil, M. R. Curado, M. Witkowski et al., “MEP predicts motor recovery in chronic stroke patients undergoing 4-weeks of daily physical therapy,” in Human Brain Mapping Annual Meeting, Beijing, China, 2012, 33WTh.

J. M. Carmena and L. G. Cohen, “Brain-machine interfaces and transcranial stimulation: future implications for directing functional movement and improving function after spinal injury in humans,” in Spinal Cord Injuries E-Book, vol. 109 of Handbook of Clinical Neurology, chapter 27, pp. 435–444, 2012.

C. R. Hema, M. Paulraj, S. Yaacob, A. H. Adom, and R. Nagarajan, “Asynchronous brain machine interface-based control of a wheelchair,” in Software Tools and Algorithms for Biological Systems, pp. 565–572, 2011.

G. Pfurtscheller, G. R. Müller, J. Pfurtscheller, H. J. Gerner, and R. Rupp, “‘Thought’—control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia,” Neuroscience Letters, vol. 351, no. 1, pp. 33–36, 2003.

A. H. Do, P. T. Wang, A. Abiri, C. King, and Z. Nenadic, “Brain-computer interface controlled functional electrical stimulation system for ankle movement,” Journal of NeuroEngineering and Rehabilitation, vol. 8, no. 1, article 49, 2011.

M. Tavella, R. Leeb, R. Rupp, and J. D. R. Millán, “Towards natural non-invasive hand neuroprostheses for daily living,” in Proceedings of the 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '10), pp. 126–129, September 2010.

A. R. Murguialday, J. Hill, M. Bensch et al., “Transition from the locked in to the completely locked-in state: a physiological analysis,” Clinical Neurophysiology, vol. 122, no. 5, pp. 925–933, 2011.

J. Clausen, “Ethische Aspekte von Gehirn-Computer-Schnittstellen in motorischen Neuroprothesen,” International Review of Information Ethics, vol. 5, pp. 25–32, 2006.

J. Clausen, “Man, machine and in between,” Nature, vol. 457, no. 7233, pp. 1080–1081, 2009.

J. L. Collinger, B. Wodlinger, J. E. Downey et al., “High-performance neuroprosthetic control by an individual with tetraplegia,” The Lancet, vol. 381, no. 9866, pp. 557–564, 2013.

S. T. Grafton and C. M. Tipper, “Decoding intention: a neuroergonomic perspective,” NeuroImage, vol. 59, no. 1, pp. 14–24, 2012.

A. Presacco, L. W. Forrester, and J. L. Contreras-Vidal, “Decoding intra-limb and inter-limb kinematics during treadmill walking from scalp electroencephalographic (EEG) signals,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 20, no. 2, pp. 212–219, 2012.

T. J. Bradberry, R. J. Gentili, and J. L. Contreras-Vidal, “Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals,” The Journal of Neuroscience, vol. 30, no. 9, pp. 3432–3437, 2010.

N. Birbaumer, G. Gallegos-Ayala, M. Wildgruber, S. Silvoni, and S. R. Soekadar, “Direct brain control and communication in paralysis,” Brain Topography. In press

S. R. Soekadar and N. Birbaumer, “Improving the efficacy of ipsilesional brain-computer interface training in neurorehabilitation of chronic stroke,” in Brain-Computer Interface Research: A State-of-the-Art Summary, C. Guger, B. Allison, and G. Edlinger, Eds., Springer, 2013.