MENU: Home Bio Affiliations Research Teaching Publications Videos Collaborators/Students Contact FAQ ©2007-14 RSS

Kihwan Kim’s Thesis Defense (2011): “Spatio-temporal Data Interpolation for Dynamic Scene Analysis”

December 6th, 2011 Irfan Essa Posted in Computational Photography and Video, Kihwan Kim, Modeling and Animation, Multimedia, PhD, Security, Visual Surviellance, WWW No Comments »

Spatio-temporal Data Interpolation for Dynamic Scene Analysis

Kihwan Kim, PhD Candidate

School of Interactive Computing, College of Computing, Georgia Institute of Technology

Date: Tuesday, December 6, 2011

Time: 1:00 pm – 3:00 pm EST

Location: Technology Square Research Building (TSRB) Room 223

Abstract

Analysis and visualization of dynamic scenes is often constrained by the amount of spatio-temporal information available from the environment. In most scenarios, we have to account for incomplete information and sparse motion data, requiring us to employ interpolation and approximation methods to fill for the missing information. Scattered data interpolation and approximation techniques have been widely used for solving the problem of completing surfaces and images with incomplete input data. We introduce approaches for such data interpolation and approximation from limited sensors, into the domain of analyzing and visualizing dynamic scenes. Data from dynamic scenes is subject to constraints due to the spatial layout of the scene and/or the configurations of video cameras in use. Such constraints include: (1) sparsely available cameras observing the scene, (2) limited field of view provided by the cameras in use, (3) incomplete motion at a specific moment, and (4) varying frame rates due to different exposures and resolutions.

In this thesis, we establish these forms of incompleteness in the scene, as spatio- temporal uncertainties, and propose solutions for resolving the uncertainties by applying scattered data approximation into a spatio-temporal domain.

The main contributions of this research are as follows: First, we provide an effi- cient framework to visualize large-scale dynamic scenes from distributed static videos. Second, we adopt Radial Basis Function (RBF) interpolation to the spatio-temporal domain to generate global motion tendency. The tendency, represented by a dense flow field, is used to optimally pan and tilt a video camera. Third, we propose a method to represent motion trajectories using stochastic vector fields. Gaussian Pro- cess Regression (GPR) is used to generate a dense vector field and the certainty of each vector in the field. The generated stochastic fields are used for recognizing motion patterns under varying frame-rate and incompleteness of the input videos. Fourth, we also show that the stochastic representation of vector field can also be used for modeling global tendency to detect the region of interests in dynamic scenes with camera motion. We evaluate and demonstrate our approaches in several applications for visualizing virtual cities, automating sports broadcasting, and recognizing traffic patterns in surveillance videos.

Committee:

  • Prof. Irfan Essa (Advisor, School of Interactive Computing, Georgia Institute of Technology)
  • Prof. James M. Rehg (School of Interactive Computing, Georgia Institute of Technology)
  • Prof. Thad Starner (School of Interactive Computing, Georgia Institute of Technology)
  • Prof. Greg Turk (School of Interactive Computing, Georgia Institute of Technology)
  • Prof. Jessica K. Hodgins (Robotics Institute, Carnegie Mellon University, and Disney Research Pittsburgh)
AddThis Social Bookmark Button

Paper ISMAR 2009: “Augmenting Aerial Earth Maps with Dynamic Information”

October 20th, 2009 Irfan Essa Posted in Computational Journalism, Computational Photography and Video, Kihwan Kim, Modeling and Animation, Papers No Comments »

Augmenting Aerial Earth Maps with Dynamic Information

  • K. Kim, S. Oh, J. Lee, and I. Essa (2009), “Augmenting Aerial Earth Maps with Dynamic Information,” in Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2009. [PDF] [WEBSITE] [VIDEO] [DOI] [BLOG] [BIBTEX]
    @inproceedings{2009-Kim-AAEMWDI,
      Author = {K. Kim and S. Oh and J. Lee and I. Essa},
      Blog = {http://prof.irfanessa.com/2009/10/20/augearth-ismar2009/},
      Booktitle = {Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
      Date-Modified = {2012-04-09 23:23:56 +0000},
      Doi = {10.1109/ISMAR.2009.5336505},
      Month = {October},
      Pdf = {http://www.cc.gatech.edu/~irfan/p/2009-Kim-AAEMWDI.pdf},
      Title = {Augmenting Aerial Earth Maps with Dynamic Information},
      Url = {http://www.cc.gatech.edu/cpl/projects/augearth/},
      Video = {http://www.youtube.com/watch?v=TPk88soc2qw},
      Year = {2009},
      Bdsk-Url-1 = {http://www.cc.gatech.edu/cpl/projects/augearth/},
      Bdsk-Url-2 = {http://dx.doi.org/10.1109/ISMAR.2009.5336505}}

Abstract

We introduce methods for augmenting aerial visualizations of Earth (from tools such as Google Earth or Microsoft Virtual Earth) with dynamic information obtained from videos. Our goal is to make Augmented Earth Maps that visualize the live broadcast of dynamic sceneries within a city. We propose different approaches to analyze videos of pedestrians and cars, under differing conditions and then augment Aerial Earth Maps (AEMs) with live and dynamic information. We also analyze natural phenomenon (clouds) and project information from these to the AEMs to add the visual reality.

For Journal Version of this paper, please see http://prof.irfanessa.com/2011/02/02/vr-2011/

AddThis Social Bookmark Button

Paper (2009) In IEEE Transactions on Visualization and CG “Fluid Simulation with Articulated Bodies”

June 10th, 2009 Irfan Essa Posted in Greg Turk, Modeling and Animation, Nipun Kwatra No Comments »

Nipun Kwatra, Chris Wojtan, Mark Carlson, Irfan A. Essa, Peter J. Mucha, Greg Turk (2009), “Fluid Simulation with Articulated Bodies“, IEEE Transactions on Visualization and Computer Graphics, 10 Jun. 2009. IEEE computer Society Digital Library. IEEE Computer Society. [DOI | PDF (see copyright) | Video | Website]

Abstract

We present an algorithm for creating realistic animations of characters that are swimming through fluids. Our approach combines dynamic simulation with data-driven kinematic motions (motion capture data) to produce realistic animation in a fluid. The interaction of the articulated body with the fluid is performed by incorporating joint constraints with rigid animation and by extending a solid/fluid coupling method to handle articulated chains. Our solver takes as input the current state of the simulation and calculates the angular and linear accelerations of the connected bodies needed to match a particular motion sequence for the articulated body. These accelerations are used to estimate the forces and torques that are then applied to each joint. Based on this approach, we demonstrate simulated swimming results for a variety of different strokes, including crawl, backstroke, breaststroke and butterfly. The ability to have articulated bodies interact with fluids also allows us to generate simulations of simple water creatures that are driven by simple controllers.

teaser

AddThis Social Bookmark Button

Paper (2009) In ACM Symposium on Interactive 3D Graphics “Human Video Textures”

March 1st, 2009 Irfan Essa Posted in ACM SIGGRAPH, Computational Photography and Video, James Rehg, Matt Flagg, Modeling and Animation, Papers, Sing Bing Kang No Comments »

 

Matthew FlaggAtsushi Nakazawa, Qiushuang Zhang, Sing Bing Kang, Young Kee Ryu, Irfan EssaJames M. Rehg (2009), Human Video Textures In Proceedings of the ACM Symposium on Interactive 3D Graphics and Games 2009 (I3D ’09), Boston, MA, February 27-March 1 (Fri-Sun), 2009 [PDF (see Copyright) | Video in DiVx | Website ]

Abstract

This paper describes a data-driven approach for generating photorealistic animations of human motion. Each animation sequence follows a user-choreographed path and plays continuously by seamlessly transitioning between different segments of the captured data. To produce these animations, we capitalize on the complementary characteristics of motion capture data and video. We customize our capture system to record motion capture data that are synchronized with our video source. Candidate transition points in video clips are identified using a new similarity metric based on 3-D marker trajectories and their 2-D projections into video. Once the transitions have been identified, a video-based motion graph is constructed. We further exploit hybrid motion and video data to ensure that the transitions are seamless when generating animations. Motion capture marker projections serve as control points for segmentation of layers and nonrigid transformation of regions. This allows warping and blending to generate seamless in-between frames for animation. We show a series of choreographed animations of walks and martial arts scenes as validation of our approach.

Example Image from Project

Human Video Textures (Output Rendered as a Collage!)

AddThis Social Bookmark Button

Paper: IEEE CVPR (2006) Element-Free Elastic Models for Volume Fitting and Capture”

June 14th, 2006 Irfan Essa Posted in Greg Turk, Modeling and Animation, Papers, Research No Comments »

Element-Free Elastic Models for Volume Fitting and Capture (IEEEXplore)

Jaeil Choi Szymczak, A. Turk, G. Essa, I.
Georgia Institute of Technology
This paper appears in: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on
Publication Date: 2006
Volume: 2
On page(s): 2245 – 2252
ISSN: 1063-6919
ISBN: 0-7695-2597-0
Digital Object Identifier: 10.1109/CVPR.2006.110
Posted online: 2006-10-09 11:11:24.0

Abstract

We present a new method of fitting an element-free volumetric model to a sequence of deforming surfaces of a moving object. Given a sequence of visual hulls, we iteratively fit an element-free elastic model to the visual hull in order to extract the optimal pose of the captured volume. The fitting of the volumetric model is acheived by minimizing a combination of elastic potential energy, a surface distance measure, and a self-intersection penalty for each frame. A unique aspect of our work is that the model is mesh free – since the model is represented as a point cloud, it is easy to construct, manipulate and update the model as needed. Additionally, linear elasicity with rotation compensation makes it possible to handle local deformations and large rotations of body parts much more efficiently than other volume fitting approaches. Our experimental results for volume fitting and capture in a multi-view camera setting demonstrate the robustness of element-free elastic models against noise and self-occlusions.

AddThis Social Bookmark Button

Thesis: Gabriel Brostow’s PhD (2004): “Novel Skeletal Representation for Articulated Creatures”

April 9th, 2004 Irfan Essa Posted in Activity Recognition, Gabriel Brostow, Modeling and Animation, Research, Thesis No Comments »

Gabriel Brostow (2004), “Novel Skeletal Representation for Articulated Creatures” PhD Thesis, Georgia Institute of Technology, College of Computing. (Advisor: Irfan Essa) [PDF] [URI]

Abstract

This research examines an approach for capturing 3D surface and structural data of moving articulated creatures. Given the task of non-invasively and automatically capturing such data, a methodologyand the associated experiments are presented, that apply to multiview videos of the subjects motion. Our thesis states: A functional structure and the timevarying surface of an articulated creature subject are contained in a sequence of its 3D data. A functional structure is one example of the possible arrangements of internal mechanisms (kinematic joints, springs, etc.) that is capable of performing the motions observed in the input data. Volumetric structures are frequently used as shape descriptors for 3D data. The capture of such data is being facilitated by developments in multi-view video and range scanning, extending to subjects that are alive and moving. In this research, we examine vision-based modeling and the related representation of moving articulated creatures using Spines. We define a Spine as a branching axial structure representing the shape and topology of a 3D objects limbs, and capturing the limbs correspondence and motion over time. The Spine concept builds on skeletal representations often used to describe the internal structure of an articulated object and the significant protrusions. Our representation of a Spine provides for enhancements over a 3D skeleton. These enhancements form temporally consistent limb hierarchies that contain correspondence information about real motion data. We present a practical implementation that approximates a Spines joint probability function to reconstruct Spines for synthetic and real subjects that move. In general, our approach combines the objectives of generalized cylinders, 3D scanning, and markerless motion capture to generate baseline models from real puppets, animals, and human subjects.

AddThis Social Bookmark Button

Thesis: Irfan Essa’s MS Thesis (1990): “Contact detection, collision forces and friction for physically based virtual world modeling”

May 3rd, 1990 Irfan Essa Posted in Masters, Modeling and Animation, Thesis No Comments »

Contact detection, collision forces and friction for physically based virtual world modeling

  • I. Essa (1990), “Contact Detection, Collision Forces and Friction for Physically Based Virtual World Modeling,” Master Thesis, Massachusetts Institute Technology, 1990. [WEBSITE] [BLOG] [BIBTEX]
    @mastersthesis{1990-Essa-CDCFFPBVWM,
      Author = {I. Essa},
      Blog = {http://prof.irfanessa.com/1990/05/03/ms-thesis-1990/},
      Date-Modified = {2011-12-13 14:50:49 +0000},
      Month = {June},
      School = {Massachusetts Institute Technology},
      Title = {Contact Detection, Collision Forces and Friction for Physically Based Virtual World Modeling},
      Url = {http://dspace.mit.edu/handle/1721.1/14054?mode=simple&submit_simple=Show+simple+item+record},
      Year = {1990},
      Bdsk-Url-1 = {http://dspace.mit.edu/handle/1721.1/14054?mode=simple&submit_simple=Show+simple+item+record}}

Abstract

Detection of contact and calculation of collision forces is an important problem in any kind of physical multi-body simulation. For computer graphics and physically based animation it is especially important to devise methods that combine efficient computational methods with powerful existing graphics tools if one is to obtain a realistic, real-time virtual world. Most of physical simulations are computationally expensive, and thus, it is difficult to set up any simulations that are stable and have a real-time response.

Efficient methods for contact detection and response for physical interactions of deformable objects in physically based virtual world environments are presented. Contact, collision and friction of objects in virtual worlds are specifically addressed in the framework of differential geometry and finite element modeling. A statistical approach is introduced for estimation and control of the physical simulation. These methods employ statistical estimation of contact between stochastically defined surfaces and linear control theory for estimation and control to obtain stable forward time simulations. By mapping from the statistical domain to the geometric domain and then to the physical domain, we have been able to obtain efficient physical simulations of multi-body systems.

AddThis Social Bookmark Button