Toward Electrocorticographic Control of a Dexterous Upper Limb Prosthesis

By Matthew S. Fifer, Soumyadipta Acharya, Heather L. Benz, Mohsen Mollazadeh, Nathan E. Crone, and Nitish V. Thakor

NOTE: This is an overview of the entire article, which appeared in the January/February 2012 issue of the IEEE Pulse magazine.
Click here to read the entire article.

Introduction

One of the most exciting and compelling areas of research and development is building brain-machine interfaces (BMIs) for controlling prosthetic limbs. Prosthetic limb technology is advancing rapidly, and the modular prosthetic limb (MPL) of the Johns Hopkins University/ Applied Physics Laboratory (JHU/APL) permits actuation with 17 degrees of freedom in 26 articulating joints. There are many signals from the brain that can be leveraged, including the spiking rates of neurons in the cortex, electrocorticographic (ECoG) signals from the surface of the cortex, and electroencephalographic (EEG) signals from the scalp. Unlike microelectrodes that record spikes, ECoG does not penetrate the cortex and has a higher spatial specificity, signal-to-noise ratio, and bandwidth than EEG signals. The authors have implemented an ECoG-based system for controlling the MPL in the Johns Hopkins Hospital Epilepsy Monitoring Unit, where patients are implanted with ECoG electrode grids for clinical seizure mapping and asked to perform various recorded finger or grasp movements. They have shown that low-frequency local motor potentials (LMPs) and ECoG power in the high gamma frequency (70-150 Hz) range correlate well with grasping parameters, and they stand out as good candidate features for closed-loop control of the MPL.

An estimated number of 541,000 Americans were living with some form of upper limb loss in 2005, and this number is projected to more than double with an aging and growing population by 2050. Loss of limb may occur congenitally or due to cancer, diseases of the vasculature, or trauma, including industrial or farming accidents and battlefield injuries. The recent wars in Iraq and Afghanistan have resulted in a large veteran population with substantial upper limb loss due to trauma. This population has inspired research in the development of advanced prosthetic limbs.

Neural Data Acquisition

The BMI community has explored many avenues of access to neural signals for BMI applications, but, traditionally, four modalities dominate: 1) EEG is the measure of neural potentials arising from the cortex from electrodes placed on the scalp, 2) ECoG is the measure of cortical potentials from the surface of the cortex, 3) local field potentials are the low-pass filtered (e.g., 200 Hz) electrical potentials recorded from cortex-penetrating microelectrodes, and 4) single or multiunit recordings to detect action potentials (or spikes) from neighboring neurons. Considering the potential strengths and weaknesses associated with these methods, ECoG occupies a unique middle ground among these technological tradeoffs.

JHU/APL MPL and an ECoG BMI schematic. (a) A photograph of the JHU/APL MPL. (b) The configuration depicted involves acquisition of the ECoG signals from electrodes placed on a human brain (left and top) and their computational analysis and modeling (right) to drive a prosthetic limb (bottom). The left and bottom figures have been adapted from screenshots of MSMS.
JHU/APL MPL and an ECoG BMI schematic. (a) A photograph of the JHU/APL MPL. (b) The configuration depicted involves acquisition of the ECoG signals from electrodes placed on a human brain (left and top) and their computational analysis and modeling (right) to drive a prosthetic limb (bottom). The left and bottom figures have been adapted from screenshots of MSMS.

Click to enlarge

System Implementation

The system the authors have developed, and continue to refine, is designed to enable communication and synchronization of three distinct nodes. In general terms, these nodes are responsible for (a) neural signal acquisition and processing, (b) behavioral kinematic acquisition, and (c) artificial limb actuation.

Neural signal acquisition is accomplished using a Neuroscan SynAmps2 hardware that can be used to amplify either EEG or ECoG signals. Behavioral kinematic data acquisition is accomplished using the Optotrak system and CyberGlove. Artificial limb actuation is achieved either in three-dimensional virtual or physical space. JHU/APL has previously reported and demonstrated MPL, a 27 degree of freedom prosthetic arm, complete with control of the shoulder, elbow, wrist, and fingers. This arm has been duplicated as a virtual model in the musculoskeletal modeling software (MSMS) simulation environment, which has been developed at the University of Southern California and is freely available online. The computational resources necessary to process the incoming neural and kinematic data are contained within a single eight-core Dell Workstation with 32-GB RAM, of which four are dedicated to MATLAB’s parallel computing toolbox.

Results

The system described has been used to initiate research into ECoG-based control of a dexterous prosthetic limb. ECoG electrode grids are predominantly implanted for clinical purposes in patients with uncontrollable epileptic seizures. They have previously discovered that the LMP recorded from subjects implanted with ECoG grids could be used to decode slow grasping motions of the hand with simple linear models. Peak decoding performance was achieved with as few as four electrodes in areas that can be intraoperatively identified as having motor involvement, meaning that these signals can be recorded from low-footprint ECoG grids implanted in known areas.

In a more recent work from the authors’ laboratory, they investigated the neural signals responsible for the coordination of slightly more complex grasps. The study showed that frequency components in the high gamma band (70-100 Hz and 100-150 Hz) provide the best performance for decoding grasp aperture.

Future Directions

Steady progress is being made toward the dream of neural control of prosthetic limbs using a variety of means, but the journey is just beginning. A few major challenges in achieving ECoG-based control of dexterous prosthetic remain, including the following:

  • improving the resolution of ECoG arrays (high-resolution ECoG with arrays of mini- and microelectrodes) may provide better localization to the areas of the cortex that are responsible for dexterous hand and finger movements
  • maturation of decoding algorithms specifically suited to ECoG signals (signals in very low frequency as well as high gamma bands) may offer novel decoding capabilities and information
  • provision of proprioceptive and touch feedback to the neuroprosthetic user (by stimulating intact peripheral nerves or directly stimulating the somatosensory cortex) may greatly facilitate natural control of an artificial limb
  • building fully implanted ECoG systems (long-term cortically controlled prosthetics) composed of electrodes, circuits, and telemetry interface to the limb while being fully implanted and powered
  • considerations in the selection of patients and implantation with regard to the potential risks and benefits to each individual patient

ABOUT THE AUTHORS

Matthew S. Fifer (msfifer@gmail.com), Soumyadipta Acharya (acharya@jhu.edu), Heather L. Benz (benz@jhu.edu), Mohsen Mollazadeh (mohsenm@jhu.edu), and Nitish V. Thakor (nthakor@jhu.edu) are with the Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, Maryland.

Nathan E. Crone (necrone@jhmi.edu) is with the Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, Maryland.