fisheye tracking and clustering

Introduction

This project originates from automated game evaluation.
A fisheye camera is used to capture a room where
people are playing a newly designed social game.
In the gram, two teams compete with each other and
the team dancing in higher sync win the game.

In the previous work, people are detected and marked with a bounding
box as in Fig. 1. In the work, we track and clustering people
into groups. People in each group should have similar behavior
and group is the minimal study unit in the following game evaluation.

Tracking

The basic idea of tracking is searching for the most similar boundary
box in the coming frame. However, in fisheye video, as people move,
the bounding boxes undergo significant deformation and it needs to be
taken into consideration when comparing box contents. We handle it by
reconstructing the real scene from the fisheye video based on fisheye
imaging model and using persisting physical constants as the clues
to predict the box deformation. Of course, it is really possible
to reconstruct the real scene from a single fisheye camera. Strong
assumption about the real scene is used which is enough for the
tracking task.

Clustering

After tracking, the trajectory of each people is obtained.
Meanshift is then used for clustering, using people location and
motion energy spectrum as features.

Back to Yuan's homepage