A video-based markerless body machine interface: a pilot study

Regaining functional independence plays a crucial role to improve the qualify of life of individuals with motor disabilities. Here, we address this problem within the framework of Body-Machine Interfaces (BoMIs). BoMIs enable individuals with restricted mobility to extend their capabilities by mapping their residual body movements into commands to control an external device. In this study, we propose a video-based marker-less interface that can track the position of the shoulders and the head using a state-of-the-art approach relying on the DeepLabCut (DLC) architecture. The high-dimensional body signal is then mapped into a lower dimensional space via non-linear variational autoencoder to obtain commands for a 2D computer cursor. First, we perform an offline test to evaluate the prediction power of the DLC fine tuned model. Then, we verify whether the proposed pipeline can be used to control a computer cursor in real-time. Results showed that the network can accurately predict the position of body landmarks. Moreover, an unimpaired participant was able to efficiently operate the computer cursor and gain a high-level of control skill after training with the interface. This enables performing experiments with video-based marker-less BoMIs for future implementation of an assistive device for people with motor disabilities.