Navigating a Virtual Helicopter

Advances in Neurotechnology Lead to ‘Thought-Driven’ Navigation of Virtual Helicopter in 3-D Space

Advances in Neurotechnology Lead to ‘Thought-Driven’ Navigation of Virtual Helicopter in 3-D Space

A helicopter in a virtual world can be controlled by ‘thought,’ as decoded from electrical signals generated by the brain. Recent work by Dr. Bin He and his group at the University of Minnesota demonstrated such possibility. In the 1982 movie Firefox, a fighter jet’s weapons were controlled by thought. Although the movie was science fiction, brain-computer interfaces (BCIs) are approaching such capabilities. Using invasive recordings, BCIs have allowed paralyzed humans to operate a computer and monkeys to select keys, move a computer cursor, and control a prosthetic arm to feed themselves. Non-invasive systems, such as those based on scalp-recorded electroencephalography (EEG), have been explored by a number of investigators to control a computer cursor, send e-mails, and various other tasks with limited capabilities.

Navigating a Virtual Helicopter

Navigating a Virtual Helicopter

Click to enlarge

Recently, a group of investigators, led by Dr. Bin He, Distinguished McKnight University Professor and Director of Center for Neuroengineering at the University of Minnesota, demonstrated for the first time that human subjects are able to continuously control the flight of a virtual helicopter navigating through the 3-dimensional virtual campus of the University of Minnesota. The figure shown on the left illustrates the 3 experiments performed, including (a) starting from a fixed position and navigating the helicopter through each of the 8 fixed rings in the sky; (b) navigating continuously through rings that were located at 8 fixed positions in the sky, and (c) navigating through rings that were generated randomly in the sky. He and co-workers recorded EEG activity while subjects imagined moving specific parts of their body. Such motor imagination modulates the rhythmic activity of the sensorimotor areas of the brain, which leads to detectable signals that are recorded by a ‘thinking’ cap (EEG cap) placed over the scalp of subjects. The EEG signals were amplified, analog-to-digital converted, and processed to extract control signals, which were fed back to control the flight of a virtual helicopter in real time. Subjects watched the flight of the virtual helicopter and performed navigation by generating appropriate control signals via motor imagery.

Amplified EEG signals

Amplified EEG signals

Click to enlarge

Brain computer interface is a progressive topic that has been researched for clinical applications in order to aid and restore functions in disabled patients. In addition, this work suggests the application to enhance functions of healthy subjects in their daily life. By interfacing the human brain with a machine, such neural interfacing technology opens avenues for duplicating humans’ ability to accomplish tasks which would otherwise need muscular control, or extending our ability to control external devices by ‘thought’. The figure shown above illustrates a comparison between brain control (left column) and keyboard control (right column; when subjects control helicopter using keyboard of a computer). From this figure it is clear that the result of brain control of flight through 8 paths is well consistent with the result of keyboard control.

The work was published in the leading neuroengineering journal, IEEE Transactions on Neural Systems and Rehabilitation Engineering as a cover article (1).

Click here to see the video of ‘thought’ driven navigation of a virtual helicopter.

Dr Bin He, IEEE Fellow and AIMBE Fellow, is the Co-Chair of the Life Sciences New Initiative Project Team, and the 2009-2010 IEEE EMBS President. He is the Distinguished McKnight University Professor, Professor of Biomedical Engineering and Director of Center for Neuroengineering at the University of Minnesota.

Hide References
  1. Royer AS, Doud AJ, Rose ML, He B: “EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies,”IEEE Trans Neural Syst Rehabil Eng. 18(6):581-589, 2010.