A Parallel Algorithm Framework for Feature Extraction of EEG Signals on MPI

In this paper, we present a parallel framework based on MPI for a large dataset to extract power spectrum features of EEG signals so as to improve the speed of brain signal processing. At present, the Welch method has been wildly used to estimate the power spectrum. However, the traditional Welch method takes a lot of time especially for the large dataset. In view of this, we added the MPI into the traditional Welch method and developed it into a reusable master-slave parallel framework. As long as the EEG data of any format are converted into the text file of a specified format, the power spectrum features can be extracted quickly by this parallel framework. In the proposed parallel framework, the EEG signals recorded by a channel are divided into N overlapping data segments. Then, the PSD of N segments are computed by some nodes in parallel. The results are collected and summarized by the master node. The final PSD results of each channel are saved in the text file, which can be read and analyzed by Microsoft Excel. This framework can be implemented not only on the clusters but also on the desktop computer. In the experiment, we deploy this framework on a desktop computer with a 4-core Intel CPU. It took only a few minutes to extract the power spectrum features from the 2.85 GB EEG dataset, seven times faster than using Python. This framework makes it easy for users, who do not have any parallel programming experience in constructing the parallel algorithms to extract the EEG power spectrum.


Introduction
EEG is a recorded signal of electrical activity of the brain, which is collected from the scalp through electrodes. e application of EEG has important practical value in medical treatment, military, sports, and the intelligence fields, which has been widely recognized by all the researchers. So far, scientists from various disciplines have achieved good results in this field. For instance, scientists from American Wadsworth Centre help the paralyzed people input 36 characters via signals instead of using their own fingers. ese signals correspond to specific brain activity. In China, scientists from Tsinghua University have designed an automatic dialling system. e system is connected to the computer for real-time dialling by interpreting the thinking mode of the brain as corresponding numbers [1][2][3][4][5][6]. e research of pattern recognition of the EEG signal includes the following steps: data collection, data storage, data processing, data classification, and recognition as shown in Figure 1.
In order to record EEG signals, several electrodes need to be placed on the scalp. Traditional device usually has 20 electrodes. However, recently, EEG devices with as many as 256 electrodes have been used, as shown in Figure 2(a). e increase in the number of electrodes enables recording of huge data thereby making the data processing stage in Figure 1 more important and complicated. is not only consumes a lot of computer resources but also leads to poor data extraction thereby directly affecting the accuracy of classification.
ere are many signal processing methods which can be used to extract EEG features with good discrimination. ese methods include time domain analysis, frequency domain analysis, and time-frequency analysis. A comparative analysis of different approaches to spectral signal representation was performed in the paper [7]. ese approaches include power spectral density (PSD) techniques, atomic decompositions, time-frequency (t-f ) energy distributions, and continuous and discrete wavelet approaches, from which band power features can be extracted and used in the framework of motor imagery (MI) classification. It is pointed out that, among all the feature types of EEG signals, PSD approaches demonstrate to be the most consistent, robust, and effective in extracting the distinctive spectral patterns for accurate discrimination between left and right MI-induced EEGs.
At present, many methods are used to calculate the PSD of signals.
e Welch method is one of the most popular, in which users calculate the PSD of the EEG signal in Python or Matlab environment. e function Scipy.signal.welch is used in Python and pwelch is used in Matlab. In case of small amount of data, PSD of the EEG signal can be quickly obtained by those two functions. But, from Figures 2(b) and 2(c), we can see that, with rapid developments in science and information technology and the upcoming of 5G era, wearable devices will widely be preferred. EEG signals of human activities can be collected by these devices. We can collect the current working or learning status of some specific people (such as drivers and students) via analysis of their EEG signals [8,9]. If we calculate the PSD in the Matlab or Python environment, it will take a long time which we cannot bear it. erefore, we focus on how to calculate the PSD of EEG signals by parallel approaches.
Parallel approaches can improve processing speeds but will need technologies that support distributed computations. Nowadays, there are two commonly different frameworks for big data analysis. One is Apache Spark, and the other is OpenMP/MPI. e two computational frameworks are compared and analyzed in literature [10]. Apache Spark has the advantage of good data management. OpenMP/MPI is faster than Apache Spark by an order of magnitude. Motivated by this analysis, we present a parallel framework for the large dataset to extract power spectrum features of EEG signals, which can be implemented in Linux and MPI environment. e main contributions of this paper are threefold:  (1) According to the principle of Welch algorithm, we propose a parallel framework of Welch algorithm, PFwelch, to compute PSD of EEG. e architecture of PFwelch is based on the master-slave mode. e EEG signals recorded by each channel are divided into N overlapping data segments. en, the N segments are computed by the master and slave node in parallel. e results are collected and summarized by the master node. e final Welch PSD results of each channel are saved in the text file, which can be read and analyzed by Microsoft Excel.
(2) A middle file of the specified format was used. ere are many kinds of EEG datasets in the world; different datasets have different file formats. In order to process any kind of EEG data by the proposed parallel framework, EEG data need to be converted into middle files. e relationship between the middle file and the parallel framework is depicted in Figure 3. (3) A comparative experiment was designed. In the experimental stage, we first run the function pwelch in the Matlab environment to extract PSD features from the EEG signal as a baseline for comparison and subsequently run PFwelch on the Ubuntu platform. e results show that the PFwelch have the same result as function pwelch. After this, we run the Python function Scipy.signal.welch in the same environment with PFwelch. e experimental result shows that the proposed parallel framework is 7 times faster than using Python. is paper is organized as follows. Section 2 takes a brief overview of the principle of the Welch method, and then presents a serial algorithm of the Welch method and the proposed parallel framework of the Welch method. In Section 3, we present experimental results and analysis. Discussion and conclusions are presented in Section 4.

Welch's Method.
e power spectral density (PSD) exhibits how the power is contained in a signal in the frequency domain. Welch's method and the multitaper approach have shown the best performance among the PSD estimators [11]. e Welch algorithm [12] is exhibited in Figure 4.
From Figure 4, we can demonstrate the Welch algorithm in the following mathematical form.
e input signal x[n], n � 0, 1, . . . , N − 1 is split into many overlapping segments. In most cases, an overlap of 50% is applied when the input signal is divided into segments. Let the length of each segment be L and the total number of segments be N s . e formula for the data in the ith segment is as follows: e procedure of the segmentation is illustrated in Figure 5. e relationship of the sampling length N, number of overlapping points N D , number of segments N S , and segmental length L is (

2)
A smooth window w(n) is applied to each segment. Generally, we usually use the Hamming window. e formula of the Hamming window for each segment is as follows: where n � 0, 1, 2 . . . , L − 1, L denotes the length of each segment. Figure 6 displays a 256-point Hamming window in the time domain and frequency domain with Matlab. e purpose of the window function is to prevent the spectral leakage [13]. Figure 7(a) shows the spectrum leakage of the original signal. Figure 7(b) exhibits that the spectrum leakage can effectively be reduced by the Hamming window.
From formula (3), we can get formula (4) for the ith segment of data after being windowed: Fourier transform of each windowed segment is computed. e formula is as follows: where A i is the Fourier transform result of the ith windowed segment, i � 0, 1, . . . , L − 1. e periodogram of each windowed segment is computed by using the following formula: where U � (1/L) L− 1 n�0 w 2 (n) denotes the mean power of the window w(n).
So, LU � L− 1 n�0 w 2 (n) denotes the energy of the window function w(n) with length L.
Finally, we can get the PSD by the Welch method which is the average of those periodograms, i.e.,

Serial Algorithm of the Welch Method.
In order to design a good parallel program, it is necessary to understand the traditional serial algorithm. According to the above description of the Welch method, the serial algorithm of the Welch method is given as Algorithm 1.

Proposed Parallel Framework of the Welch Method
According to the steps of serial algorithms of the Welch method in Section 2.2, it can be seen that this algorithm can be implemented in parallel with MPI. e structure of the parallel algorithm is master-slave and is demonstrated as Figure 8. e structure of PFWelch algorithm in Section 2.2 contains the following stages: input, split, map, reduce, and output stage. One channel data are split into seven segments, which are allocated to four nodes according to certain rules. All nodes are responsible for computing. In the reduce stage, the master process is responsible for receiving the result from slave processes and compute the final PSD. Since the parallel implementation of fast Fourier transform has been maturely developed [14][15][16][17][18][19][20], we need not give the relevant details. e parallel algorithm of the Welch method is described in Algorithm 2.  x 0 e key point of the parallel program is how to cooperate with each node [21]. How to distribute tasks evenly to each computing node is one of the major factors affecting the performance of the parallel program. From Figure 8, the EEG data of each channel are divided into 7 segments (n_segs � 7). e length of each segment is 64. e allocation scheme is shown in Table 1 when the number of nodes is 4 (size � 4). According to Table 1, it is convenient to calculate the signal range that each processor needs to compute. For example, processor 0 needs to handle segments 0 and 4. From formula (1), it can be calculated that the signal range is [0, 63] and [128,191], respectively. Computational and Mathematical Methods in Medicine 5

Dataset.
In the experiment, the data were downloaded from the website http://kdd.ics.uci.edu/databases/eeg/eeg. html. is dataset arises from a large study to examine EEG correlates of genetic predisposition to alcoholism. It includes measurements from 64 electrodes placed on the scalp sampled at 256 Hz. ere are three versions of the EEG data set: small data set, large data set, and full data set. e full data set contains all 120 trials for 122 subjects. e entire set of data is compressed to about 700 MB [22]. When it was uncompressed, its size is approximately 2.8 GB. After all files were converted into middle files, there were in all 11,058 files.

Testing Method and Environment.
ere are three different performances which need to be tested: (1) Accuracy. e result of MATLAB is compared with that of the parallel framework.
(2) Speed. e running time of Python program is compared with that of the parallel framework.
(3) Speedup. e parallel framework was run with different nodes. e running time was recorded, and speedup can be obtained. e testing environment of hardware is shown in Table 2, and the testing environment of software is shown in Table 3.

Experimental Results
. PSD computation of one EEG channel was conducted in the Matlab by using the Welch function, i.e., pwelch (data, hamming(64), 32, 64, 256). e pwelch parameters mean that sample data are split up into segments, each segment has 64 data points, the overlap of neighboring segments is 32, Hamming window is applied, and the sampling rate is 256 Hz. Figure 9 shows the result of the Welch method in three different window functions in Matlab. index++; (8) end for (9) for seg_start in range (1, x_len-seg_len + 1) (10) seg_end � seg_start + seg_len − 1; //seg_start and seg_end is start point and end point of each data segment, respectively. (11) apply Hamming window to the data of welch [n] between [seg_start, seg_end]; (12) Do FFT for windowed data. e result is stored in variable fft_x; (13) Calculating the square of the amplitude of the fft_x. e result is stored in variable pgram; (14) Summate power spectra Pxy + � pgram; (15) start_seg + � seg_len-overlap; (16) end for (17) Wel_psd � Pxy/(n_ffts × Fs × u); ALGORITHM 1: Serial algorithm of the Welch method. We performed the same test in the parallel framework. e PSD results were stored in the text file which can be accessed by Microsoft Excel and is shown in Figure 10.
Comparison between Figures 9 and 10 indicates that the results obtained by both Matlab and PFWelch are consistent, which corroborates the correctness of PFWelch. To test the time performance, we first used different number of nodes in different segments to calculate the PSD of one EEG file with PFWelch (in this experiment, we regard different CPU cores as different nodes). Time consumption (measured in seconds) is recorded in Table 4. For comparison, the time cost in Python environment by the serial algorithm is also listed in Table 4. As we have not used Matlab Linux version, we carried out such comparative experiments with Python.
Second, we use different number of nodes with 7 segments to calculate the PSD of all EEG files with PFWelch. e time cost is recorded in Table 5. Speedup can evaluate the time performance of PFWelch. e definition of speedup is speedup � Ts/Tp, where Ts indicates the time of serial operations and Tp indicates the time of parallel operations. e speedups are shown in Figure 11. Figure 11 demonstrates that, as the number of nodes increases, the speedup also increases. But, in different cases, the increase is not the same. For all EEG files, Figure 11 shows that, as the number of nodes increases to a certain value, the speedup slowly increases. e reason is the time it takes includes files open and close.
is time cannot be changed when the number of nodes is increased.
For a single EEG file, the PFWelch shows the best performance when the signal is split into 7 segments. is is because it has the maximum speedup when the number of nodes is 4. erefore, we divided the signal into 7 segments to calculate the PSD of all EEG files in Figure 11. If the signal is split into 15 segments, the speedup grows basically in a linear manner, but its speedup is the lowest. Figure 12 clearly reveals the relationship between the number of segments and the speedup.
It can be seen that, by increasing the number of nodes, the speedup is improved. But, merely increasing the number of nodes cannot improve the speedup. From Figure 12 Calculate how many segments: n_segs (5) for node � 1 to size par-do//Parallel process (6) apply Hamming window to welch [n] //n between [startPos, stopPos]; startPos and stopPos is starting point and end point of certain data segment, respectively. (7) Do FFT for windowed data. e result is stored in variable fft_x; (8) Calculating the square of the amplitude of the fft_x. e result is stored in variable pgram; (9) Summate power spectra Pxy + � pgram; (10) end for (11) if (rank � � 0)//master process (12) Receive Pxy from each process and compute average of all periodogram. (13) else//slave processes (14) Send Pxy to master process END ALGORITHM 2: Parallel algorithm of the Welch method.   Computational and Mathematical Methods in Medicine 7 can see that the speedup decreases when the number of segments increases to 15. When the number of segments is 7 and the number of nodes is 4, we get the best speedup performance.

Discussion and Conclusion
At present, although there are several methods which can extract features from the EEG signal, the PSD is still one of the most important methods. But, while dealing with a large      number of EEG data, it is necessary to improve the calculation speed of PSD. erefore, the parallel framework proposed in this paper is used to solve the problem of taking long time to extract PSD features from the EEG dataset in big data environment. e framework is based on C + MPI language and is completed by the master-slave mode. Compared with the traditional serial Welch method, this framework divides the signal into N segments and distributes them evenly to different nodes on which PSD can be calculated in parallel. In the experiment, the values of N are 3, 7, and 15, respectively. e number of nodes is from 1 to 4. For the given data set, we found that, although the speedup can be improved by increasing the nodes, the speedup performance will decrease if there are too many segments. Experiments show that the best performance of the parallel framework can be achieved only when the number of nodes and segments is reasonably selected. e speed of PFWelch is 7 times faster than using Python in the same hardware and operating system platform.
Because of the powerful function of MPI, this framework can be deployed not only on the cluster but also on the desktop computer, which is very convenient for the users.
e experimental results also corroborate that the framework is correct, efficient, and has a good practical value. It can be applied to extract all kinds of EEG datasets with a little modification. Researchers who are interested in PFWelch can download the source code from https://github.com/ abcxq.

Data Availability
All the data utilized in our research can be accessed from http://kdd.ics.uci.edu/databases/eeg/eeg.html.

Conflicts of Interest
e authors declare that they have no conflicts of interest.