Affordance Segmentation Using RGB-D Sensors for Application in Portable Embedded Systems

Depth sensors play a major role in the control pipeline of semi-autonomous prostheses. The feed from these sensors can complement, or even substitute, the information retrieved by RGB cameras. The paper explores the application of depth sensors to affordance detection, to recognize the graspable part of an object in foreground images. The experiments confirm that the depth (“D”) components can boost the accuracy in predicting the affordable parts of an object, when they are matched with state-of-the-art computer vision architectures. A prototype of a portable inference system with real-time performances has been assembled using a Jetson Tx2 and a Realsense D435i camera, confirming the proposal’s suitability for semi-autonomous prostheses.