Copyright © 2009 Elsevier Ltd All rights reserved.
Available online 7 August 2009.
Brain-machine interfaces (BMIs) can be characterized by the technique used to measure brain activity and by the way different brain signals are translated into commands that control an effector. We give an overview of different approaches and focus on a particular BMI approach: the movement of an artificial effector (e.g. arm prosthesis to the right) by those motor cortical signals that control the equivalent movement of a corresponding body part (e.g. arm movement to the right). This approach has been successfully applied in monkeys and humans by accurately extracting parameters of movements from the spiking activity of multiple single-units. Here, we review recent findings showing that analog neuronal population signals, ranging from intracortical local field potentials over epicortical ECoG to non-invasive EEG and MEG, can also be used to decode movement direction and continuous movement trajectories. Therefore, these signals might provide additional or alternative control for this BMI approach, with possible advantages due to reduced invasiveness.
Keywords: Directional tuning; Decoding; SUA; MUA; LFP; ECoG; EEG; MEG; BMI; BCI
- 1. Introduction
- 1.1. Brain-machine interfaces
- 1.2. Different BMI approaches
- 2. Directional tuning
- 2.1. Directional tuning in spiking signals: SUA and MUA
- 2.2. Amplitude spectrograms of analog population signals
- 2.3. Directional tuning in analog population signals: LFP, ECoG, EEG, MEG
- 3. Movement decoding
- 3.1. Movement decoding using analog population signals: movement direction
- 3.2. Movement decoding using analog population signals: movement trajectories
- 4. Discussion
- 4.1. Relevance of low-pass filtered analog neural population signals
- 4.2. Directional tuning of signals reflecting a wide range of spatial scales: from SUA to EEG/MEG
- 4.3. Online direct motor BMI and adaptivity
- Appendix A
- A.1. Deduction of the expected amplitude of a cosine created by the sum of cosines with random phases
- A.2. Deduction of the noise amplitude resulting from summation of noise
Fig. 1. General scheme of a brain-machine interface (BMI) for restoration of motor control: while in healthy people the output of neural activity in the motor cortex is conveyed to motor neurons in the spinal cord and from there to the muscles, ultimately controlling a limb, this connection might be interrupted due to injuries or neuro-degenerative diseases. A BMI reads the activity of neuronal ensembles and transmits it to a computer program, which then interprets this activity as commands to control an effector (e.g. an artificial limb).
Fig. 2. Schematic overview of the recording techniques SUA/MUA, LFP, ECoG, and MEG/EEG showing (from top to bottom): the spatial scale at which signals are recorded, the characteristic of the signal (discrete vs. analog), and the correspondence to different scales.
Fig. 3. Different paradigms to investigate directional tuning. (A) Center-out movements: movements (usually hand and/or arm) are performed from a central starting point to one of several targets around the origin. (B) Movement trajectories: movements are performed freely, to an arbitrary target, or as a pursuit in following a moving target. Velocity vectors and/or current position are estimated at successive times dense enough to represent the trajectory satisfyingly. Decoding then refers to a regression of continuous variables.
Fig. 4. Classical direct motor BMI approach based on single-unit activity (SUA) and population vector decoding. (A) From Georgopoulos et al. (1982): variation of the discharge of a motor cortical cell with different movement directions in two dimensions. For each movement direction (center) five trials (in surrounding plots) are shown, aligned to movement onset. Dashes indicate single spikes. (B) From Georgopoulos et al. (1988): example of population coding of movement direction in three dimensions. Blue lines (reflecting preferred directions of each cell scaled by firing rates) represent the vectorial contributions of individual cells in the population. The movement direction vector is shown in yellow and the direction of the population vector in red. (C) From Taylor et al. (2002). Reprinted with permission from AAAS: application of the three-dimensional population vector approach to control a prosthetic device in real-time. A monkey’s brain-controlled trajectories are shown for movements from the center to one of eight targets located on the corners of a cube (trained target, subplots A and B) or for movements to one out of six untrained targets on the center points of the faces of a cube (subplots C and D).
Fig. 5. Grand-average time-resolved amplitude spectrograms during center-out movements for the different recording techniques (LFP, ECoG, EEG, MEG). Spectrograms depict trial-average across all movement directions averaged across multiple subjects. Recording sites were in or above the motor cortex contra-lateral to center-out hand/arm movements. LFP recordings from motor cortex of monkeys (taken from Rickert et al. (2005)), ECoG recordings from primary motor cortex of epilepsy patients (taken from Ball et al. (2009)), EEG (re-analyzed from Waldert et al. (2008)) and MEG (taken from Waldert et al. (2008)) recorded simultaneously from motor areas from healthy subjects.
Fig. 6. Directionally tuned, low-pass filtered movement-related potentials (MRPs) in different types of population signals. (A) Trial-averaged (black) and single-trial (gray) monkey LFP for eight different directions of center-out movements, aligned on movement onset. (B) Average human ECoG from one electrode on hand/arm motor cortex, measured during continuous target-to-target movement and sorted for eight different instantaneous movement directions (lower plot). The vertical solid line shows the time of a new target appearance (t = 0) while the dotted line indicates the median time of target reaching. Colored bands display the mean over all single traces of one direction ± standard-error of the mean. The upper inset shows the average magnitude of hand velocity. (C) Averaged MRP recorded simultaneously with one EEG electrode and one MEG sensor (D), from Waldert et al. (2008), above the contra-lateral motor area of one subject (average ± standard-error of the mean across all trials for each direction, blue – right, green – up, red – left, cyan – down).
Fig. 7. Comparison of different recording techniques with respect to the decoding accuracy (DA) and the decoded information (DI) about movement direction for center-out experiments. The gray curves reflect the dependency of DI on DA under certain simplifying assumptions (Eq. (2), Section 3.1) for 3, 4, or 8 targets. (squares – Mehring et al., 2003; star – Ball et al., 2009; circles – Waldert et al., 2008 (contra-lateral: only sensors above contra-lateral motor areas were used, otherwise sensors above contra- and ipsilateral motor areas were used); triangles – Hammon et al., 2008 (plan/move refers to decoding of brain activity during planning/movement phase)).
Corresponding authors. Address: Faculty of Biology, Hauptstr. 1, 79104 Freiburg, Germany. Tel.: +49 761 203 2542; fax: +49 761 203 2921 (S. Walbert), tel.: +49 761 203 2543; fax: +49 761 203 2921 (C. Mehring).
1 These authors contributed equally to the manuscript.