. Research Summary
My research topic is brain machine interface (BMI). Brain machine interface is a technology for paralyzed people who cannot move their arms due to damage to their bodies from an accident or a disease. The main goal is to allow paralyzed people to interact with society more freely by giving them control over an external device such as a robot arm or a mouse cursor from the brain signals through a mathematical model. Briefly, the ultimate goal of the field is to make a cyborg that merges a human’s brain with a machine’s body like Robocop. The achievements I have done until now can be broadly divided into three topics. The first is implementation of a wheelchair for totally paralyzed people that can be controlled by EEG signals from the brain. The second is reconstruction of arm movement in four degrees of freedom by estimating the EMG signals of arm muscles from the neural signals measured in the primary motor cortex. The third is the implementation of a new wheelchair system for paralyzed people who cannot use a normal joystick, freely controlled by EMG signals from the user’s neck and arm muscles.
1. Real-Time Control of a Wheelchair by EEG Signals
In this study, we made a wheelchair system for totally paralyzed people from the neck down that can be controlled by EEG signals from the brain. If a person imagines clenching the left hand, the oscillations of the mu (8-13 Hz) and beta (14-30 Hz) rhythm of the right part of the sensorimotor cortex decrease. Conversely, if a person imagines squeezing the right hand, the oscillations decrease on the left part of the sensorimotor cortex. In foot imagery, the oscillations decrease in the foot area of the sensorimotor cortex. By using this phenomenon (ERD), it is possible to control an object by extracting commands from brain signals.
The wheelchair system implemented in this study consists of several processing blocks. It contains the signal acquisition block that is used to measure the EEG signals from the brain. It also contains a signal preprocessing block for artifact rejection and noise reduction. We used AMUSE method. For extracting the features of the signals, it contains spatial-frequency filters that can efficiently detect the ERD phenomenon. Linear support vector machines are then used to classify the signals after extracting the features. The classified signals are then transformed to the command output that is used to control the wheelchair. Finally, the wheelchair movement provides feedback to the user. The trained subjects could freely control a wheelchair with multi degrees of freedom and requiring fast response time. Further, we succeeded in controlling the wheelchair system by using the P300 paradigm.
2. Research for Brain Machine Interfaces (BMIs) using EMG signals
The methodology of this study is first estimation of EMG signals of arm muscles from the neural signals of the primary motor cortex in the brain using a linear regression method, and then reconstruction of arm movement in four degrees of freedom in the shoulder and elbow from the estimated EMG signals using a modular artificial neural network model. Here, the reason we used EMG signals is to reconstruct not only position information ,but also force information such as joint torque and stiffness. And, since anatomically muscle is linked to the primary motor cortex by 2 or 3 neurons through the spinal cord, we can get a signal that is highly correlated with the primary motor cortex.
In this study, first we trained a Japanese monkey to do a continuous arm reaching task that consisted of pushing buttons continuously. After training the monkey, we measured neural activities from 105 neurons in the primary motor cortex by using the conventional single unit recording method. The main recording target was the neurons located in layer V. At the same time, EMG signals were measured in nine muscles related to four degrees of freedom. We rectified the obtained EMG signals, then low pass filtered at 3Hz by using 4th order butterworth filter. This signal we get as a result is very similar to the actual tension, hence, it is called quasi-tension. And, by attaching an infrared sensor to the shoulder, the elbow and the wrist of the monkey, we measured each position of them, and then calculated the joint angles by using the inverse kinematics equations.
The estimation of 9 muscle tensions is obtained by simply linearizing the neural activities with the linear regression method. Delay time from neural activity to EMG signals was obtained by the ICMS, that is, we stimulated primary motor cortex by electricity and measured the output time of EMG signals. As a result, the delay time was 16.57m seconds. And then, we used the modular artificial neural network model to calculate four joint angles of the arm from the estimated muscle tensions. The reason we used the modular artificial neural network is, in the case of isotonic movement where force is outputted with changing length of the muscle, the tension is different depending on the velocity that the muscle flexes or extends. In the case that a muscle flexes, the tension will decrease as the flex velocity increases. And, in the case that muscle extends, the tension will increase as the extension velocity increases. We could improve the estimation performance of the joint angles by training two networks with tension values, which changes depending on the velocity rather than training the data in the same network. We used one network for 0 velocity and the other for movement velocity.
Finally, we reconstructed arm movement in four degrees of freedom resulting in a correlation coefficient of 0.92 with the actual arm movement. The reconstruction of arm movement through EMG signals is the first in this field and is already published in the IEEE EMBS.
3. Development of a New, Human-Centered Wheelchair System Controlled by EMG Signals
In this study, first we changed the existing joystick operation method to new joystick operation method that is more intuitive. In the existing joystick mapping method, it uses the polar coordinate system, hence, predicting the position to move to and the operation order for moving to such positions was very difficult. In particular, in the case of the backward operation, it is opposite to the direction of an automobile and a bicycle that is familiar to humans. We changed the existing joystick mapping method to the new joystick mapping method that can predict the position to move to and the operation order for moving to such positions, easily. And, we conducted an experiment to evaluate which method is more intuitive. From the result of the experiment, we knew that the new joystick mapping method has better performance. Using the new mapping method that considers intuitiveness, we designed a wheelchair system for paralyzed people who can not use the joystick, that can be controlled by the EMG signals from a user’s arm and neck. To use this wheelchair system simply by every user, we used the simple algorithm does not necessary train the data. In fact, since we tested this wheelchair system on a pedestrian road, all users could use the wheelchair simply. This wheelchair system was exhibited in the national museum of emerging science and innovation in Japan and the study result was published in the IEEE IJCNN and the IEICE transaction, etc.
