Craniux: A LabVIEW-Based Modular Software Framework for Brain-Machine Interface Research

This paper presents “Craniux,” an open-access, open-source software framework for brain-machine interface (BMI) research. Developed in LabVIEW, a high-level graphical programming environment, Craniux offers both out-of-the-box functionality and a modular BMI software framework that is easily extendable. Specifically, it allows researchers to take advantage of multiple features inherent to the LabVIEW environment for on-the-fly data visualization, parallel processing, multithreading, and data saving. This paper introduces the basic features and system architecture of Craniux and describes the validation of the system under real-time BMI operation using simulated and real electrocorticographic (ECoG) signals. Our results indicate that Craniux is able to operate consistently in real time, enabling a seamless work flow to achieve brain control of cursor movement. The Craniux software framework is made available to the scientific research community to provide a LabVIEW-based BMI software platform for future BMI research and development.


Enforcement of deterministic engine execution
In general, there is little ambiguity in the order of execution of core operations as the data-driven nature of LabVIEW programming dictates that execution follows the availability of data. Furthermore, if more explicit control of execution is desired, a number of LabVIEW components can be used to accomplish this. During the development of Craniux software, we have taken advantage of LabVIEW's standard mechanisms to ensure deterministic execution through the use of sequence structures, queues, and other synchronization elements. These actions are explained in detail below.

Main Sequence Execution
Deterministic execution of the main sequence loop in an engine is driven by incoming TCP data or the availability of hardware-acquired data in conjunction with a sequence structure. In the first frame of the structure, engine execution is halted until an incoming packet of data is received from the previous engine. Once this data is parsed and written to local controls (i.e. a variable in LabVIEW with an associated GUI element on the VI front panel) within the engine, execution proceeds to the next frame in the sequence. It is here that engine-specific implementation code is to be placed. Within this sequence frame the automatic parallelism inherent to LabVIEW is the most useful, as the execution of the current engine, and subsequently the execution of all other engines, is not allowed to proceed until all operations present in this sequence frame are complete. Once this occurs, execution proceeds to a subsequent sequence frame, where data to be transmitted to the next engine is packaged up and transmitted via TCP. Additionally, transmission of data to be saved is transmitted to the data saving manager and visualization data to the engine's associated GUI occurs within this frame. In short, this execution can be summarized in the following sequential steps, with parallel dataflow-driven execution occurring within each step: 1. Wait for incoming TCP data 2. Perform some engine-specific operation 3. Package up and send outgoing TCP data, save data, and send visualization data to GUI.

Updating of Parameters by GUIs
Parameter updates are triggered by user input on the front panel of an engine's GUI. Value change events for all GUI parameters are monitored within the GUI; updated parameter values are packaged up and sent via TCP to the engine as soon as a value change event is triggered. Within the associated engine, a TCP listener waits for incoming parameter updates in a process operating in parallel to that of the main sequence loop (see above). Once a parameter update packet is received, this packet is placed into a queue that is read immediately following transmission of data to the next engine (step '3', described in the preceding section). A semaphore keeps parameter values from updating during execution of the 'enginespecific operation' frame (step '2', above). This ensures that instances where parameter values are read multiple times within the engine-specific operation frame use the same parameter value, eliminating race conditions.

Sending of Visualization Data
Sending of visualization data occurs during sequence frame (3) identified above. Here, the values of all variables to be transmitted to the GUI for visualization are read and packaged into a single TCP packet. This packet is placed into a single-element queue retaining the most recent packet (a 'lossy' queue), which is read by a sending process running in parallel to execution of the main sequence. This packet is read from the queue when it becomes available and then sent via TCP to the user interface. In instances where the transfer of a packet is not performed before the next packet is available, the previous packet is overwritten by the new packet, ensuring only the most recent visualization data is sent to the GUI. The user interface then parses these packets and writes visualization data to the appropriate GUI variables. From here, raw visualization data is processed as needed and updated on the GUI.

Data Saving
Data is initially saved in LabVIEW's TDMS format. To specify what data is saved, the developer must create two string arrays in each engine's initialization frame. The first string array specifies the names of 'sampled variables' that should be saved for every sample of neural data that is processed. The second array is names of 'saved controls,' which are engine parameters that should only be saved when their value is changed during run-time. The developer also has the option of automatically including all variables that aren't in the sampled variables list as saved controls. During initialization, arrays of references to these two sets of variables are created. A reference to the data packet number is included in both groups so that the experiment can be reconstructed afterwards with the data properly aligned in time.
The values of sampled variables are placed in a queue after the engine completes its main execution frame. The values of saved controls are placed in the queue after one of them is changed. In parallel to the main execution loop, values are pulled from the buffer and sent via the TCP framework to a data saving manager on the user interface host. This module formats and streams the data to the TDMS file. Each channel of streamed data is notated by the variable name, the engine it came from, and whether it is a sampled variable or saved control. Array size information is also saved to ensure correct reconstruction of each sample from the streams of data. A single TDMS file is saved for each experimental run; stopping or suspending system execution closes all references to the current data file.
A separate LabVIEW VI has been created to convert Craniux TDMS files into the MATLAB (Mathworks, Inc.) MAT format. These MAT files contain a data structure with 6 sub-structures: one for each engine type (signal source, signal processing, or application) paired with each data type (sampled variables and controls). Each channel of scalar data in the TDMS file is converted to an array, with the data packet number providing the time index for each element, and placed in the proper sub-structure. Each channel of array data is parsed using the corresponding size data. Since the size of the array could be dynamic, the data is placed in a cell array with each cell corresponding to a sample of the array for each data packet number. Currently only 1D and 2D arrays can be saved in this way.

Transfer of information between engines
As mentioned in the manuscript, transfer of information between engines occurs via TCP. Names of variables to be transmitted from one engine to another are defined in a string array located in an engine's initialization procedure. Based on these names, an array of references to the desired variables is generated; this array is used when sending data from one engine to the next. Within the third sequence frame of the main sequence defined previously (see 'Main Sequence Execution'), this array of references is used to obtain the current values of the specified variables to be transmitted. Two arrays are then generated: an array of strings representing the names of the variables being transmitted, and an array of type 'variant' data containing the value of each of these variables. These two arrays are then packaged into a single TCP packet and transmitted to the next engine in the loop.
This transmitted packet is then read and reconstructed into a pair of name/value arrays in the same format as sent from the previous engine. Then, for each variable name, the corresponding variable value is written to the variable on the current engine that has the same name and data type. If no such variable is found, no action is taken by the system. In this manner, a developer must ensure that transmitted variables are consistent between engines.
When receiving signals via UDP from external hardware the Craniux system will wait indefinitely for the next packet of data. Currently, there is no timeout check for incoming data, though this could easily be added to the software. Parallel processing is used to ensure incoming data is continuously saved to disk without loss. In this manner, the real-time processing chain is always operating with the most recent block of data.