Regaining functional independence plays a crucial role to improve the qualify of life of individuals with motor disabilities. Here, we address this problem within the framework of Body-Machine Interfaces (BoMIs). BoMIs enable individuals with restricted mobility to extend their capabilities by mapping their residual body movements into commands to control an external device. In this study, we propose a video-based marker-less interface that can track the position of the shoulders and the head using a state-of-the-art approach relying on the DeepLabCut (DLC) architecture. The high-dimensional body signal is then mapped into a lower dimensional space via non-linear variational autoencoder to obtain commands for a 2D computer cursor. First, we perform an offline test to evaluate the prediction power of the DLC fine tuned model. Then, we verify whether the proposed pipeline can be used to control a computer cursor in real-time. Results showed that the network can accurately predict the position of body landmarks. Moreover, an unimpaired participant was able to efficiently operate the computer cursor and gain a high-level of control skill after training with the interface. This enables performing experiments with video-based marker-less BoMIs for future implementation of an assistive device for people with motor disabilities.

A video-based markerless body machine interface: a pilot study

Moro M.;Odone F.;Casadio M.
2021

Abstract

Regaining functional independence plays a crucial role to improve the qualify of life of individuals with motor disabilities. Here, we address this problem within the framework of Body-Machine Interfaces (BoMIs). BoMIs enable individuals with restricted mobility to extend their capabilities by mapping their residual body movements into commands to control an external device. In this study, we propose a video-based marker-less interface that can track the position of the shoulders and the head using a state-of-the-art approach relying on the DeepLabCut (DLC) architecture. The high-dimensional body signal is then mapped into a lower dimensional space via non-linear variational autoencoder to obtain commands for a 2D computer cursor. First, we perform an offline test to evaluate the prediction power of the DLC fine tuned model. Then, we verify whether the proposed pipeline can be used to control a computer cursor in real-time. Results showed that the network can accurately predict the position of body landmarks. Moreover, an unimpaired participant was able to efficiently operate the computer cursor and gain a high-level of control skill after training with the interface. This enables performing experiments with video-based marker-less BoMIs for future implementation of an assistive device for people with motor disabilities.
978-3-030-68789-2
978-3-030-68790-8
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1070884
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact