Non-intrusive tracking of multiple users in a spatially immersive display

Abstract

We present a novel vision-based system for tracking multiple users in a spatially immersive display. Without requiring them to wear any markers or other devices, we can detect and track the heads of several participants. In a projection-based display environment, the lighting conditions make it difficult to extract silhouettes or shape features from acquired images. Using a separate IR lighting and stereo camera system solves the problem, and makes background subtraction simple and fast. We start by finding general location of the users' heads in each image, from the silhouettes and projection histogram of the foreground regions. These points are used to create search areas, one in each image of a stereo pair. By cross-correlation between the search areas, corresponding points in each image are identified, and these are used to determine an accurate 3D location on the head. Finally, the search areas in consecutive frames are correlated to maintain the identification of the users over time. Experimental results demonstrate the viability of the proposed system.