Two-person Interaction Detection Using Body-Pose Features and Multiple Instance Learning

Presented at The 2nd International Workshop on Human Activity Understanding from 3D Data at Conference on Computer Vision and Pattern Recognition, CVPR 2012
Kiwon Yun, Jean Honorio, Debaleena Chattopadhyay, Tamara L. Berg, Dimitris Samaras
Stony Brook University

Abstract

Human activity recognition has potential to impact a wide range of applications from surveillance to human computer interfaces to content based video retrieval. Recently, the rapid development of inexpensive depth sensors (eg. Microsoft Kinect) provides adequate accuracy for real-time full-body human tracking for activity recognition applications. In this paper, we create a complex human activity dataset depicting two person interactions, including synchronized video, depth and motion capture data. Moreover, we use our dataset to evaluate various features typically used for indexing and retrieval of motion capture data, in the context of real-time detection of interaction activities via Support Vector Machines (SVMs). Experimentally, we find that the geometric relational features based on distance between all pairs of joints outperforms other feature choices. For whole sequence classification, we also explore techniques related to Multiple Instance Learning (MIL) in which the sequence is represented by a bag of body-pose features. We find that the MIL based classifier outperforms SVMs when the sequences extend temporally around the interaction of interest.

Publications

  • Kiwon Yun, Jean Honorio, Debaleena Chattopadhyay, Tamara L. Berg, and Dimitris Samaras, The 2nd International Workshop on Human Activity Understanding from 3D Data at Conference on Computer Vision and Pattern Recognition, CVPR 2012 (Rhode Island/USA)
    [PDF]
    • @inproceedings{kiwon_hau3d12,
      title={Two-person Interaction Detection Using Body-Pose Features and Multiple Instance Learning},
      author={Kiwon Yun and Jean Honorio and Debaleena Chattopadhyay and Tamara L. Berg and Dimitris Samaras},
      booktitle={Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on}, year={2012},
      organization={IEEE}
      }
  • Supplementary Materials [PDF]
  • Poster [PDF]
  • Talk slides [PDF]

Download

SBU Kinect Interaction Dataset (3.3GB)

Acknowledgements

This work was supported in part by NSF Awards IIS-1111047 and IIS-1054133.