Detection of 3D position of eyes through a consumer RGB-D camera for stereoscopic mixed reality environments

A novel approach to track the 3D position of the user's eyes in stereoscopic virtual environments, where stereo glasses are worn, is proposed. Such an approach improves a state-of-the-art real-time face tracking algorithm by addressing the occlusion due the stereo glasses and providing estimation of eye position based on biometric features. More generally, our solution can be seen as a proof of concept for a more robust approach to improving motion tracking techniques. In particular, the proposed technique yields accurate and stable estimates of the 3D position of the user's eyes, while the user moves in front of the stereoscopic display. The correct tracking of both eyes' 3D position is a crucial step in order to achieve a more natural human-computer interaction which diminishes visual fatigue. The proposed approach is validated through quantitative tests: (i) we assessed the accuracy of our algorithm for tracking the 3D position of users' eyes with and without stereo glasses; (ii) we have performed a perceptual assessment of the natural interaction in the virtual environments through experimental sessions with several users.