The so-called amyotrophic lateral sclerosis (ALS) or motor neuron disease (MND) is a neurodegenerative disease with various causes. It is characterized by muscle spasticity, rapidly progressive weakness due to muscle atrophy, and difficulty in speaking, swallowing, and breathing. The severe disabled always have a common problem that is about communication except physical malfunctions. The steady-state visually evoked potential based brain computer interfaces (BCI), which apply visual stimulus, are very suitable to play the role of communication interface for patients with neuromuscular impairments. In this study, the entropy encoding algorithm is proposed to encode the letters of multilevel selection interface for BCI text input systems. According to the appearance frequency of each letter, the entropy encoding algorithm is proposed to construct a variable-length tree for the letter arrangement of multilevel selection interface. Then, the Gaussian mixture models are applied to recognize electrical activity of the brain. According to the recognition results, the multilevel selection interface guides the subject to spell and type the words. The experimental results showed that the proposed approach outperforms the baseline system, which does not consider the appearance frequency of each letter. Hence, the proposed approach is able to ease text input interface for patients with neuromuscular impairments.
The neurodegenerative disease or spinal cord injury always reduces the muscle ability and results in difficulties in speaking, swallowing, and breathing. The life quality of people with the above disease is always greatly reduced. Therefore, it would make them find difficulty in performing even simple activities, such as communicating with other people. Although these severely disabled lost many human functions, they still have healthy eyes and brain which can help them see and think. In order to better provide the needs of these severely disabled, a lot of high technology assistive devices have been developed in recent years. There is an interactive method by brain computer interface (BCI) technology proposed in this paper to help the disabled communicate with other people.
A BCI connects human brain to computer or other devices by attaching electrodes on the human scalp. In general, we can acquire the electroencephalography (EEG) and extract several features from the EEG signals by using BCI technology. The BCI technology will help people to send messages or commands to the outside world or devices without saying any word [
Previous studies on BCI topic use event-related potential (ERP) method associated with either physical or mental occurrence in time. Event-related potential often includes the
Recently, the BCI technology is utilized in mental text input systems by using different methods of EEG recognition, such as P300, motor imagery, and steady-state evoked potential (SSVEP) [
An entropy encoding algorithm is a lossless data compression scheme and it could assign a unique prefix-free code to each unique symbol. The length of each code word is approximately proportional to the negative logarithm of the probability. Therefore, the most common symbols use the shortest codes [
In this study, a multilevel selection interface using entropy encoding algorithm is applied to a SSVEP-based BCI text input system. In order to improve the accuracy of SSVEP-based BCI, a multilevel selection interface is proposed to reduce the candidates of a decision model. For the purpose of improving the efficiency of multilevel selection interface, the letters are arranged to fit a tree structure by using an entropy encoding algorithm, which considers the appearance frequency of each letter. The Bayesian decision model with Gaussian mixture model is adopted to classify frequency responses.
The rest of this paper is organized as follows. Section
In this section, a SSVEP-based BCI text input system is presented to improve performance beyond that of previously proposed methods. First, the power spectrum of EEG is extracted and used to represent the evoked response of a user. Second, the letters are optimally arranged as a tree structure and used to design the multilevel selection interface. Finally, the Bayesian decision model with Gaussian mixture model is used to decide the correct selection. The procedure of proposed approach is described in the following subsections. Feature extraction is presented in Section
In this subsection, the features used in this study are presented. For a EEG signal,
After removing the drift phenomenon of
To estimate the evoked response, the power spectrum of
The procedure to design a multilevel selection interface is illustrated in this subsection. The frequencies of used letters, which were calculated from several large-scale English corpora, can be found in previous study [
For multilevel selection interface, there are 5 block options used to represent the selections. In these options, 4 options are used to select a letter or a group of letters. The 5th option is “SPACE” and “BACK command” for the first and other level, respectively. The “BACK command” enables a user to back a previous level when he/she makes a wrong selection. An example of multilevel selection interface is shown in Figure
An example of multilevel selection interface.
The purpose of allocating the letter position into the optimal option block can effectively reduce the selecting times of multilevel selection interface and then the entropy encoding algorithm is able to find an optimal allocation of letters. Thus, an entropy encoding algorithm proposed by Huffman is integrated to find an assignment of letter and is denoted by EEA. In this approach,
Furthermore, Navarro and Brisaboa had shown that the average length of codes for an encoding algorithm can be effectively reduced by adding dummy letters with zero probability. Therefore, the dummy letters with zero probability are considered in this study (denoted by EEA_D) and then the number of letters modeled by EEA_D (denoted by
For an input EEG signal, a power spectrum,
In this study, a Gaussian probability density function in
To evaluate the proposed approach, the number of options used in multilevel selection interface is set to be 5. Besides, the letters used in this SSVEP-based BCI text input system can be categorized into four groups, including 26 uppercase letters, 26 lowercase letters, 10 numbers (from 0 to 9), and 4 special letters (comma, dot, question mark, and exclamation mark). The subjects were visually stimulated by using an LCD screen and the visual stimulator flickering at five frequencies from 6 Hz to 10 Hz with 1 Hz increment. Then, the EEG signals were measured using electrodes placed at the Oz, A1, and A2 (ground) in accordance with the international EEG 10–20 system. The sampling rate and the frame size were set to be 1 k Hz and 1000, respectively. Besides, a 2nd order band-pass filter with cut-off frequencies between 5 and 30 Hz is used to limit the frequency range of the EEG signals to the subject’s responses on visual stimulation.
To evaluate the performance of different multilevel selection interface, the average message length is adopted and it can be derived as
In this study, the arrangement of letters can be modeled as a tree structure. Therefore, an approach, which does not consider the appearance frequency of each letter, was treated as a baseline system and was used to compare with the proposed approaches EEA and EEA_D. In the baseline system as shown in Figure
The codes of letters for baseline, EEA, and EEA_D multilevel selection interface were shown in Tables
The encoding results of baseline system.
Letter | Code | Letter | Code | Letter | Code |
---|---|---|---|---|---|
A | 000 | W | 112 | s | 222 |
B | 001 | X | 113 | t | 223 |
C | 010 | Y | 120 | u | 232 |
D | 011 | Z | 121 | v | 233 |
E | 002 | a | 130 | w | 300 |
F | 003 | b | 131 | x | 301 |
G | 012 | c | 122 | y | 310 |
H | 013 | d | 123 | z | 311 |
I | 020 | e | 132 | 0 | 302 |
J | 021 | f | 133 | 1 | 303 |
K | 030 | g | 200 | 2 | 312 |
L | 031 | h | 201 | 3 | 313 |
M | 022 | i | 210 | 4 | 320 |
N | 023 | j | 211 | 5 | 321 |
O | 032 | k | 202 | 6 | 330 |
P | 033 | l | 203 | 7 | 331 |
Q | 100 | m | 212 | 8 | 322 |
R | 101 | n | 213 | 9 | 323 |
S | 110 | o | 220 | ? | 3320 |
T | 111 | p | 221 | ! | 3321 |
U | 102 | q | 230 | , | 3322 |
V | 103 | r | 231 | . | 3323 |
The encoding results of EEA.
Letter | Code | Letter | Code | Letter | Code |
---|---|---|---|---|---|
A | 11 | W | 0001 | s | 1230 |
B | 0003 | X | 01203 | t | 0333 |
C | 011 | Y | 0000 | u | 000202 |
D | 010 | Z | 000200 | v | 012001 |
E | 02 | a | 1232 | w | 01231 |
F | 120 | b | 00303 | x | 0120021 |
G | 122 | c | 00021 | y | 03311 |
H | 001 | d | 01201 | z | 0120022 |
I | 21 | e | 00323 | 0 | 0033 |
J | 03313 | f | 03310 | 1 | 0121 |
K | 0122 | g | 03312 | 2 | 0332 |
L | 002 | h | 01202 | 3 | 00301 |
M | 032 | i | 00022 | 4 | 00300 |
N | 20 | j | 000201 | 5 | 0330 |
O | 13 | k | 012000 | 6 | 00320 |
P | 121 | l | 01232 | 7 | 01230 |
Q | 000203 | m | 1233 | 8 | 00302 |
R | 23 | n | 00023 | 9 | 1231 |
S | 22 | o | 01233 | ? | 012003 |
T | 10 | p | 00322 | ! | 0120023 |
U | 031 | q | 0120020 | , | 030 |
V | 0031 | r | 00321 | . | 013 |
The encoding results of EEA_D.
Letter | Code | Letter | Code | Letter | Code |
---|---|---|---|---|---|
A | 2211 | W | 3333 | s | 20 |
B | 2332 | X | 1012210 | t | 02 |
C | 2213 | Y | 01123 | u | 012 |
D | 3313 | Z | 1012211 | v | 330 |
E | 3312 | a | 03 | w | 222 |
F | 01122 | b | 231 | x | 3331 |
G | 10120 | c | 010 | y | 220 |
H | 3330 | d | 32 | z | 10123 |
I | 2321 | e | 00 | 0 | 332 |
J | 10121 | f | 100 | 1 | 0110 |
K | 23203 | g | 103 | 2 | 1010 |
L | 01120 | h | 30 | 3 | 2330 |
M | 2212 | i | 13 | 4 | 2323 |
N | 2322 | j | 23200 | 5 | 0113 |
O | 01121 | k | 0111 | 6 | 2333 |
P | 3311 | l | 31 | 7 | 3332 |
Q | 101223 | m | 013 | 8 | 2331 |
R | 3310 | n | 12 | 9 | 2210 |
S | 1013 | o | 11 | ? | 101222 |
T | 1011 | p | 102 | ! | 1012212 |
U | 23201 | q | 23202 | , | 223 |
V | 101220 | r | 21 | . | 230 |
An example to represent a letter “D” with code “011” in tree structure.
To compare the performance of baseline system, EEA, and EEA_D, the experimental results expressed as average, minimum, and maximum length of codes were shown in Table
The experimental results expressed as average, minimum, and maximum length of codes for different approaches.
Method | Average | Minimum | Maximum |
---|---|---|---|
length | length | length | |
Baseline | 3.03 | 3 | 4 |
EEA | 2.63 | 2 | 7 |
EEA_D | 2.42 | 2 | 7 |
Next, the proposed approaches are examined by using two types of sentences which are pangram and nonpangram. A pangram is a sentence that contains all letters of the alphabet. The testing pangrams are shown as follows. Bored? Craving a pub quiz fix? Why, just come to the Royal Oak! Sphinx of black quartz, judge my vow! Just keep examining every low bid quoted for zinc etchings. How razorback jumping frogs can level six piqued gymnasts! Grumpy wizards make toxic brew for the evil queen and Jack.
The testing nonpangrams (NP) are shown as follows. Life is not about getting and having, it is about giving and being. Whatever the mind of man can conceive and believe, it can achieve. Strive not to be a success, but rather to be of value. Two roads diverged in a wood, and I took the one less traveled by, And that has made all the difference. I attribute my success to this, I never gave or took any excuse.
The experimental results for pangram and nonpangram were shown in Table
The average length of codes for pangrams and nonpangrams.
Number | Pangrams | Nonpangrams | ||||
---|---|---|---|---|---|---|
Baseline | EEA | EEA_D | Baseline | EEA | EEA_D | |
1 | 3.08 | 3.27 | 2.98 | 3.04 | 2.53 | 2.35 |
2 | 3.06 | 3.35 | 3.03 | 3.04 | 2.56 | 2.31 |
3 | 3.02 | 2.74 | 2.54 | 3.05 | 2.56 | 2.35 |
4 | 3.02 | 2.92 | 2.72 | 3.04 | 2.55 | 2.26 |
5 | 3.02 | 2.86 | 2.65 | 3.04 | 2.56 | 2.40 |
|
||||||
Average | 3.04 | 3.03 | 2.78 | 3.04 | 2.55 | 2.33 |
In this subsection, the performance of proposed SSVEP-based BCI text input system was examined. 12 subjects (10 males and 2 females), who have normal vision and had no history of any neurological or psychological disorders, were asked to participate in the experiments. The average age of the subjects is 20.9 and 21.5 years old for males and females, respectively.
In this experiment, the evoked responses of the visual stimulation from 6 Hz to 10 Hz with 1 Hz increment are used to represent the five options in multilevel selection interface. Then the Bayesian decision models were used to recognize the evoked EEG signal of the subject due to these letter options. The experimental results were shown in Figure
The experimental results of proposed SSVEP-BCI based text input system.
The examples of evoked signals for (a) 7 Hz and (b) 10 Hz.
The performance of proposed approach was examined by using the information transfer rate (ITR). The ITR used in this study is based on Wolpaw method, which is presented by Volosyak [
The ITR were 31.9 bits/min, 39.04 bits/min, and 42.62 bits/min for baseline, EEA, and EEA_D, respectively, after the experiments for the SSVEP-based BCI text input system. It is clear that the ITR can be improved by arranging the order of the letters in a multilevel selection interface. According to the entropy encoding algorithm, the input times for a letter with high appearance frequency can be effectively reduced and then the ITR can be effectively improved. In this experiment, the accuracy of an option (blinking frequency is 10 Hz) is lower than other option blocks (blinking at 6 Hz, 7 Hz, 8 Hz, and 9 Hz) and it can be regarded as an infrequent option block for allocating the letters or special marks with low appearance inside. So, the ITR is not seriously affected by the entropy encoding algorithmin this study. However, when the accuracy is quite different for each option, the ITR would be greatly affected and the accuracy should be considered in a multilevel selection interface.
In this study, the entropy encoding algorithm was proposed to improve the efficiency of multilevel selection interface for the SSVEP-based BCI text input system. The entropy encoding algorithm was successfully used to improve the efficiency of the tree structure of multilevel selection interface according to the appearance frequency of each letter. The Bayesian decision model with Gaussian mixture model was able to recognize the input options and provided the appropriate EEG recognition. The experimental results demonstrated that the proposed EEA and EEA_D, new letters arrangements in option blocks of BCI, outperform baseline arrangement. The proposed EEA and EEA_D could be applied to a practical application and improve the text input speed and performance of the SSVEP-based BCI text input system. Meanwhile, the SSVEP-based BCI text input system can solve the communication problems for the severely disabled such as amyotrophic lateral sclerosis, cerebral palsy, and spinal cord injury. In the future, the recognition accuracy of evoked EEG signal stimulated with different visual stimulation frequencies should be considered to improve the performance of entropy encoding algorithm for a multilevel selection interface.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors would like to thank the Ministry of Science and Technology of the Republic of China, Taiwan, for financially supporting this research under Contract MOST 103-2221-E-218-009 and MOST 103-2218-E-218-002.