This project proposes and delivers a novel approach to train and test a Convolutional Neural Network (CNN) model for muscle synergy controlled prosthetic hands. The project is focused on providing a solution for precise control and real-time testing of prosthetic hand control used by below-elbow amputees having independent control over the prosthetic fingers. Multiple EMG sensors that are placed on the forearm will be used to control the prosthetic hand using the trained model. CNN allows us to extract features from raw EMG signals without the requirement for manual feature engineering done over raw data in traditional methods. Furthermore, the trained model will be evaluated in real-time within a Virtual Reality environment developed using the Mujoco Physics environment with the HTC Vive VR headset. The developed algorithm will be tested on ten healthy participants and their data will be analyzed to show the performance of the presented controller.
Ninapro is a a publicly available multimodal database to foster research on robotic & prosthetic hands controlled with artificial intelligence. Ninapro includes electromyography, kinematic, inertial, eye tracking, visual, clinical and neurocognitive data. Ninapro data are used worldwide by scientific researchers in machine learning, robotics, medical and neurocognitive sciences.
InceptionTime is a 1D Convolution based Deep Neural Network model which is currently quoted as the state of the art TSC model from the published paper InceptionTime: Finding AlexNet for Time Series Classification.
The Virtual Reality environment is developed with Mujoco Physics Engine, OpenVR SDK and Delsys Trigno SDK for aquiring the EMG signals. The VR environment can be found in a specifically self-developed seperate repository. Mujoco Virtual Reality Development (https://github.com/fazildgr8/mujoco200_awear)
| Data Prep | Val Accuracy | Val Loss | Train Accuracy | Train Loss |
|---|---|---|---|---|
| Grasp window=150x8 40 subjects 9 Labels | 98.96% | 0.43 | 99.82% | 0.004 |
| Wrist window=150x8 40 subjects 2 Labels | 90% | |||