Posts Categorized:

Kinect-Wheelchair Interface Controlled (KWIC) Robotic Trainer for Powered Mobility

By Daniel K. Zondervan and David J. Reinkensmeyer

It is important that children with disabilities have the ability to learn how to use a powered wheelchair. The Kinect™-Wheelchair Interface Controlled (KWIC) Trainer is an intelligent powered wheelchair training device that can convert a manual wheelchair into a powered wheelchair and allow a caregiver to use natural gesture commands to assist a child being trained. This allows a caregiver to train any child in a safe, natural, and effective way that is engaging and intuitive for both the child and the caregiver.

The ability to move independently is critical for cognitive and social development and quality of life. A powered wheelchair can provide this ability to children with a severe disability. Unfortunately, teaching children to drive a powered wheelchair can sometimes require hours of training, and current techniques have limitations. A caregiver must typically walk alongside the child, placing her hand over the child’s on the joystick to manually guide him along and maintain safety. Hand-over-hand guidance may draw the child’s attention towards the caregiver, and away from the direction the caregiver would like the child to move. Further, if the caregiver is uncertain if the child understands the desired direction of motion, then it is difficult to gauge how much assistance to provide. Over-assisting may reinforce the child’s dependence on the caregiver and prevent the child from making and correcting errors, a crucial requirement for motor learning.

We have developed a robotic training device that uses a Microsoft Kinect™ sensor to 1) allow the caregiver to use natural gestures to provide assistance to the child, and 2) create a “virtual leash” between the caregiver and the wheelchair for safety. The Kinect-Wheelchair Interface Controlled (KWIC) Trainer (Fig. 1) consists of a powered wheelchair platform built by Dr. John Farris at Grand Valley State University that converts any standard manual wheelchair to a powered wheelchair (so that the child may use their existing seating system), a USB joystick, and a Kinect Sensor. An on-board laptop handles the control of the system. The system can also be adapted to work with switches instead of a joystick. A video of a child using the KWIC Trainer can be found here.

Figure 1: The KWIC Trainer
Figure 1:The KWIC Trainer. The child is able to use the system in his own manual wheelchair. The Kinect™ is mounted on a vertical boom so it can track objects in front the system. The USB joystick is mounted on an adjustable arm so that it can be placed in the optimal position for each child. The open hand gesture shown signals the Trainer to move forward.

Click to enlarge

To use the system, the caregiver walks in a desired path in front of the chair. The child is given full control of forward and backward movement of the wheelchair, but the caregiver is able to override this input using simple, intuitive gesture commands: opening both hands with palms upward signals the chair to drive toward the therapist, raising one hand with palm outward reverses the chair, and raising both hands with palms outward stops the chair. To control chair rotation, The KWIC Trainer uses a shared control algorithm that uses a weighted average of the joystick input from the child and the left-right position of the caregiver, as sensed by the Kinect. That is, the caregiver is like a joystick that moves when the caregiver walks left and right; this joystick-like input is blended with the chair joystick. Adjusting the level of shared control changes the maximum rotational error a child is able to make. We have also developed an adaptive control algorithm that increases the amount of control the child has of wheelchair rotation if he is driving well, and reduces it if he is driving poorly.

There are several advantages of the KWIC Trainer over traditional hand-over-hand guidance. It puts the caregiver in front of the child, at a safer and more natural focus point, and allows her to signal to the child the intended driving direction in a naturalistic way simply by moving and/or calling to the child. It allows the child to make bounded steering errors and explore the cause and effect relationship of the joystick in a controlled environment. It automatically prevents the child from steering the wheelchair away from the caregiver and into potentially hazardous situations. The result is a system that allows the caregiver and robot to work in synergy to teach wheelchair driving skills in an engaging, intuitive, and safe manner.

We will be loaning the KWIC Trainer to a local therapy clinic in order to obtain feedback on its ease of use and efficacy. This will allow us to continue to refine the design and verify that it is an appropriate tool for caregivers to use to train powered wheelchair driving skills. We also believe there is potential for the KWIC Trainer to be used as more than just a training device. The adaptive control algorithm could be used to provide a numerical assessment of a child’s driving skill, which could be important for both research purposes and pragmatic issues such as insurance reimbursement. Also, the functionality of the device could easily be used in an assistive mode, where the Kinect sensor allows a child to “lock-on” to a person they would like to approach, and the device automatically steers their wheelchair in that direction. We look forward to seeing any extensions of the work others could take forward as well.

This research was supported by National Institute of Disability and Rehabilitation Research field-initiated grant H133G090111.

About the Newsletter

The IEEE Life Sciences Newsletter is a new initiative to bring forth interesting articles and informative interviews within the exciting field of life sciences every month. Please subscribe to the Newsletter to receive notification each month when new articles are published.

November 2012 Contributors

Nitish V. ThakorNitish V. Thakor is a Professor of Biomedical Engineering at Johns Hopkins University, Baltimore, USA, as well as the Director of the newly formed institute for neurotechnology, SiNAPSE, at the National University of Singapore. Read more

Paolo BonatoPaolo Bonato, Ph.D., serves as Director of the Motion Analysis Laboratory at Spaulding Rehabilitation Hospital in Boston, MA. Read more

Skyler Ashton DalleySkyler Ashton Dalley received a B.E. degree in mechanical engineering, in 2007, from Vanderbilt University, Nashville, TN, where he is currently a Ph.D. student in mechanical engineering. Read more

Dan ZondervanDan Zondervan holds a B.S in Electrical Engineering from Calvin College, a M.S. in Mechanical Engineering from U.C. Irvine, and is currently pursuing a PhD in Mechanical Engineering. Read more

David J. ReinkensmeyerDavid J. Reinkensmeyer received a B.S. degree in electrical engineering from the Massachusetts Institute of Technology in 1988, and M.S. and Ph.D. degrees in electrical engineering... Read more