• Raffay Hamid, Siddhartha Maddi, Amos Johnson, Aaron Bobick, Irfan Essaand Charles Isbell (2009) “A novel sequence representation for unsupervised analysis of human activities” in Artificial Intelligence, Volume 173, Issue 14, September 2009, Pages 1221-1244. [PDF][DOI][Science Direct]

• Hamid, Maddi, Johnson, Bobick, Essa, and Isbell (2009), “A Novel Sequence Representation for Unsupervised Analysis of Human Activities,” Artificial Intelligence Journal, 2009. [BIBTEX]

  @article{2009-Hamid-NSRUAHA,
    Author = {R. Hamid and S. Maddi and A. Johnson and A. Bobick and I. Essa and C. Isbell},
    Date-Modified = {2011-12-08 21:27:48 +0000},
    Journal = {Artificial Intelligence Journal},
    Month = {May},
    Title = {A Novel Sequence Representation for Unsupervised Analysis of Human Activities},
    Year = {2009}}

Abstract

Formalizing computational models for everyday human activities remains an open challenge. Many previous approaches towards this end assume prior knowledge about the structure of activities, using which explicitly defined models are learned in a completely supervised manner. For a majority of everyday environments however, the structure of the in situ activities is generally not known a priori. In this paper we investigate knowledge representations and manipulation techniques that facilitate learning of human activities in a minimally supervised manner. The key contribution of this work is the idea that global structural information of human activities can be encoded using a subset of their local event subsequences, and that this encoding is sufficient for activity-class discovery and classification.

In particular, we investigate modeling activity sequences in terms of their constituent subsequences that we call event n-grams. Exploiting this representation, we propose a computational framework to automatically discover the various activity-classes taking place in an environment. We model these activity-classes as maximally similar activity-cliques in a completely connected graph of activities, and describe how to discover them efficiently. Moreover, we propose methods for finding characterizations of these discovered classes from a holistic as well as a by-parts perspective. Using such characterizations, we present a method to classify a new activity to one of the discovered activity-classes, and to automatically detect whether it is anomalous with respect to the general characteristics of its membership class. Our results show the efficacy of our approach in a variety of everyday environments.

Keywords: Temporal reasoning; Scene analysis; Computer vision

We received around 60 submissions to this track and expect a few NECTAR (new scientific and technical advances in research) submissions too (DUE Feb 18, 2008). The primary track submissions are in process of review).

Abstract Submission Deadline: January 25, 2008 *DONE*
Paper Submission Deadline: January 30, 2008 *DONE*
Author Notification Deadline: April 1, 2008

Abstract

In this paper, we present techniques for recognizing com- plex, multitasked activities from video. Visual information like image features and motion appearances, combined with domain-specific information, like object context is used ini- tially to label events. Each action event is represented with a unique symbol, allowing for a sequence of interactions to be described as an ordered symbolic string. Then, a model of stochastic context-free grammar (SCFG), which is devel- oped using underlying rules of an activity, is used to provide the structure for recognizing semantically meaningful behav- ior over extended periods. Symbolic strings are parsed us- ing the Earley-Stolcke algorithm to determine the most likely semantic derivation for recognition. Parsing substrings al- lows us to recognize patterns that describe high-level, com- plex events taking place over segments of the video sequence. We introduce new parsing strategies to enable error detection and recovery in stochastic context-free grammar and meth- ods of quantifying group and individual behavior in activities with separable roles. We show through experiments, with a popular card game, the recognition of high-level narratives of multi-player games and the identification of player strate- gies and behavior using computer vision.

Recognizing Black Jack

Abstract

AI researchers are interested in building intelligent machines that can interact with them as they interact with each other. Science fiction writers have given us these goals in the form of HAL in 2001: A Space Odyssey and Commander Data in Star Trek: The Next Generation. However, at present, our computers are deaf, dumb, and blind, almost unaware of the environment they are in and of the user who interacts with them. In this article, I present the current state of the art in machines that can see people, recognize them, determine their gaze, understand their facial expressions and hand gestures, and interpret their activities. I believe that by building machines with such abilities for perceiving, people will take us one step closer to building HAL and Commander Data.