Learning Actions from Human and Robot Visual Motion Observations

Although verbal communication is regarded as the primary mode of interaction, it is not always sufficient for effective interactions. Humans rely on a developed ability to read and convey important information without using words. Aspects such as gaze, posture, and movement can be useful indicators of the intentions or psycho-physical conditions of those in front of us. This thesis focuses on motion, trying to understand how much body movement alone can be exploited in contexts of interaction. We started from hypothetical everyday scenarios in which humans and robots interact naturally. Imagine to teach a robot how to store objects on a shelf. Some objects may be fragile. More delicate objects will be stored in the right place with smoother movements. The robot will need to be able not only to place objects in the correct place, but also to perceive the different style in which we move objects and in turn adapt its movements to the object’s properties. From these assumptions this thesis proposes imitation models that take into account not only the goal (WHAT) but also the style (HOW). Traditionally in robotics, imitation problems have always involved explicit access to the internal controls (joint angles, motor currents, etc.) of the demonstrator agent. However, this thesis has the ambition of developing interactions that are as natural as possible. For this reason we wanted to rely solely on the robot’s visual system. In the literature this approach is often referred as IFO (Imitation from observation). If we imagine a context in which two agents interact in front of each other we immediately notice two main problems: Demonstrator and imitator often have two different bodies (embodiment mismatch) and the two do not share the same perspective of the scene (perspective mismatch). This thesis proposes different ways to address both problems and offers models able to imitate goal and style of actions. The study first provides a comprehensive review of existing methods for representing and processing motion data. It then presents original contributions in the form of new algorithms and models for learning from motion. The experimental results demonstrate the effectiveness of the proposed methods, as well as their potential usefulness in the field of social robotics. Overall, the thesis provides a contribution to the field of human robot interaction and offers new insights into how intelligent and interactive systems could effectively learn from human motion.