by JOHN MECKLIN

Dec 23, 2008 in Miller-McCune

“If you pay passing attention to the media landscape, you know that most mainstream news outlets have had their business models undermined by the digital revolution. As their general-interest monopolies have been pillaged by niche online competitors, traditional news organizations have lost revenue and cachet, laying off journalists in waves that have grown into tsunamis. This process has created dire prospects for the future of investigative reporting, often seen as the most costly of journalistic forms.”

Goes on to mention Computational Journalism and our (at GA Tech) and recent Duke University’s efforts in this space and few others.

ABSTRACT

We propose the use of 3D (2D+time) Shape Context to recognize the spatial and temporal details inherent in human actions. We represent an action in a video sequence by a 3D point cloud extracted by sampling 2D silhouettes over time. A non-uniform sampling method is introduced that gives preference to fast moving body parts using a Euclidean 3D Distance Transform. Actions are then classified by matching the extracted point clouds. Our proposed approach is based on a global matching and does not require specific training to learn the model. We test the approach thoroughly on two publicly available datasets and compare to several state-of-the-art methods. The achieved classification accuracy is on par with or superior to the best results reported to date.

This academic year, I am spending some time working with the newly formed Disney Research, Pittsburgh, (Directed by Jessica Hodgins) formed next to CMU.  The press release is announcing this lab is here (Carnegie Mellon SCS Press Release). I am also hanging out with folks at the CMU Robotics Institute and have started some new collaborations.  So now depending on when, you can find me either in Atlanta (at GA Tech) or in Pittsburgh (at Disney Lab or CMU) [OR on a airplane between Pittsburgh and Atlanta].

ABSTRACT

We have developed an audio-based casual puzzle game which produces a time-stamped transcription of spokenaudio as a by-product of play. Our evaluation of the game indicates that it is both fun and challenging. The transcripts generated using the game are more accurate than those produced using a standard automatic transcription system and the time-stamps of words are within several hundred milliseconds of ground truth.

Videolyzer is a tool designed to help journalists and bloggers collect, organize, and present information about the quality (i.e. validity, reliability, etc.) of online videos. It makes it possible to evaluate and make sense of things like comments, claims, and sources as they relate to the video. Users can comment and annotate pieces of the video (called “anchors”) to provide a more fine-grained description of the information in the video. The interface also incorporates a tightly integrated transcript of what’s spoken in the video to make it easier to navigate the dense information there. Finally, Videolyzer allows for collaboration among many people. Users can build off of each other’s annotations and rate each other in a form of distributed vetting and peer-evaluation.

ABSTRACT

Using off-the-shelf Global Positioning System (GPS) units, we reconstruct buildings in 3D by exploiting the reduction in signal to noise ratio (SNR) that occurs when the buildings obstruct the line-of-sight between the moving units and the orbiting satellites. We measure the size and height of skyscrapers as well as automatically constructing a density map representing the location of multiple buildings in an urban landscape.  If deployed on a large scale, via a cellular service provider’s GPS-enabled mobile phones or GPS-tracked delivery vehicles, the system could provide an inexpensive means of continuously creating and updating 3D maps of urban environments.

ABSTRACT

Making sense of the information quality of online media including things such as the accuracy and validity of claims and the reliability of sources is essential for people to be well-informed. We are developing Videolyzer to address the challenge of information quality sense-making by allowing motivated individuals to analyze, collect, share, and respond to criticisms of the information quality of online political videos and their transcripts. In this paper specifically we present a model of how the annotation ontology and collaborative dynamics embedded in Videolyzer can enhance information quality.

 
Abstract

Digital image capture, processing, and sharing has become pervasive in our society. This has had significant impact on how we create novel scenes, how we share our experiences, and how we interact with images and videos. In this talk, I will present an overview of series of ongoing efforts in the analysis of images and videos for rendering novel scenes. First I will discuss (in brief) our work on Video Textures, where repeating information is extracted to generate extended sequences of videos. I will also describe some our extensions to this approach that allows for controlled generation of animations of video sprites. We have developed various learning and optimization techniques that allow for video-based animations of photo-realistic characters. Using these sets of approaches as a foundation, then I will show how new images and videos can be generated. I will show examples of Photorealistic and Non-photorealistic Renderings of Scenes (Videos and Images) and how these methods support the media reuse culture, so common these days with user generated content.   I will then describe some of our new efforts that move towards understanding mobile imaging and video, and also discuss issues of collaborative imaging and authoring and ad-hoc sensor networks, and peer production with images and videos, leading to a new concepts of how computation has impacted journalism. Time permitting, I will also share some of our efforts on video annotation and how we have taken some of these new concepts of video analysis to classrooms.

Investigator(s): Umakishore Ramachandran, (Principal Investigator), Irfan Essa (Co-Principal Investigator)

Dates: September 1, 2008 – August 31, 2009 (Estimated)

Abstract

From the western world to the third world, the use of handheld devices (cellphones, PDAs) has proliferated. The world of users is becoming both wireless and mobile. Web 2.0 has ushered in an age wherein the web is viewed as a provider of services and not just a repository of documents and/or information. Despite this advance, the web remains just that, a single web with an inherent assumption that a powerful computing and communication infrastructure supports it. Couldn’t mobile wireless devices in close proximity form a web of their own? This is the vision behind this project, the Web on Demand (WoD). WoD aims at bridging the gap between social networks and ad hoc networking. In other words, it aims to rethink the system software stack all the way from application to networking that would allow the creation and management of social networks without any assumption of infrastructure support. The core of the research is to develop software technologies for mobile devices that would allow the dynamic creation of thematic ad hoc overlay networks empowering (a) mobile people with similar interests (e.g., weather forecast), (b) friends and family (e.g., in a theme park), and (c) participants in mission critical applications (e.g., search and rescue), stay connected. WoD complements the World Wide Web (WWW) and leverages it when it is available, such as exploiting the ambient computing infrastructure to enhance user experience, and managing the dynamic creation of User Generated Content (UGC) by mobile users. The vision behind this project is to democratize access to services that are currently offered through WWW. In this sense, the results from this research can have far-reaching technological and societal consequences. Most importantly, the research will help breed a new class of computer scientists who are connected with societal causes in addition to advancing technology.

• Date and Time: Wednesday, 13 August 2008 | 1:45 pm – 5:30 pm
• Location: Room 502 A, Los Angeles Convention Center, Los Angeles, CA, USA

Fundamentally, journalism is the process of collecting news information and disseminating that information with a layer of contextualization and understanding provided by journalists in the form of a news story. Recent advances in computational technology are rapidly affecting how news is gathered, reported, and distributed, and how stories are authored and told. New technologies for aggregating, visualizing, summarizing, consuming, and collaborating on news are becoming increasingly popular. They are challenging the traditional practices of journalism and directly affecting the future of news production and consumption. Computation and journalism share a deep interest in information and the value it provides to society, and they are deeply involved in the future of storytelling in various contexts, especially current events. This class summarizes how these new technologies affect journalism, both at the core of the journalism discipline and in its practice and business. Topics include: the technologies that have empowered citizen journalism and related citizen media production and authoring; mobile and sensing technologies that allow journalism to become ubiquitous and pervasive; the changes in photo, video, and broadcast journalism; and how web, online, and science journalism are changing the basic processes of reporting. Instructors focus especially on areas of special interest to the SIGGRAPH community: photography and video, large-scale information visualization, and social networking.

Presentations will be made by:

• Irfan Essa (Georgia Institute of Technology) [website]
• Brad Stenger (WIRED NextFest)
• Maneesh Agrawala (University of California, Berkeley) [website]
• Jeffrey Heer (University of California, Berkeley, Stanford University) [website]
• Eric Ulken (LA Times) [website]
• Chloe Sladden (Current TV) [website]

This course is open to all registrant of ACM SIGGRAPH 2008 and has not pre-requisite requirements. See the info on ACM SIGGRAPH Site

By Nick Diakopoulos (My PhD Student)

Audio Puzzler is a new kind of puzzle game based on unauthored content found online. The audio for the puzzles is taken from popular or interesting video clips from different genres such as news, documentary, or television. The audio puzzler is the type of game that harnesses people’s play to also provide valuable data which enriches the content played with. This is in the same vein as the ESPGame, the Listen Game, and PhotoPlay, which are all games which gather data in the process of game play. But while the data collected by these other games is useful for machine learning, the data collected with audio puzzler is immediately valuable as a transcription of the speech in the video. A similar effort (but in a much grander domain) is the Fold It project which seeks to harness playtime to solve protein folding problems. Much more detailed information about the evaluation of the technology will be forthcoming in a paper to be published at ACM Multimedia in October.

M. Raffay Hamid PhD (2008), “A Computational Framework For Unsupervised Analysis of Everyday Human Activities“, PhD Thesis, Georgia Institute of Techniology, College of Computing, Atlanta, GA. (Advisor: Aaron Bobick & Irfan Essa)

Abstract

In order to make computers proactive and assistive, we must enable them to perceive, learn, and predict what is happening in their surroundings. This presents us with the challenge of formalizing computational models of everyday human activities. For a majority of environments, the structure of the in situ activities is generally not known a priori. This thesis therefore investigates knowledge representations and manipulation techniques that can facilitate learning of such everyday human activities in a minimally supervised manner. 

A key step towards this end is finding appropriate representations for human activities. We posit that if we chose to describe activities as finite sequences of an appropriate set of events, then the global structure of these activities can be uniquely encoded using their local event sub-sequences. With this perspective at hand, we particularly investigate representations that characterize activities in terms of their fixed and variable length event subsequences. We comparatively analyze these representations in terms of their representational scope, feature cardinality and noise sensitivity.

Exploiting such representations, we propose a computational framework to discover the various activity-classes taking place in an environment. We model these activity-classes as maximally similar activity-cliques in a completely connected graph of activities, and describe how to discover them efficiently. Moreover, we propose methods for finding concise characterizations of these discovered activity-classes, both from a holistic as well as a by-parts perspective. Using such characterizations, we present an incremental method to classify

a new activity instance to one of the discovered activity-classes, and to automatically detect if it is anomalous with respect to the general characteristics of its membership class. Our results show the efficacy of our framework in a variety of everyday environments

Abstract

This research addresses the problem of temporal pattern discovery in real-valued, multivariate sensor data. Several algorithms were developed, and subsequent evaluation demonstrates that they can efficiently and accurately discover unknown recurring patterns in time series data taken from many different domains. Different data representations and motif models were investigated in order to design an algorithm with an improved balance between run-time and detection accuracy. The different data representations are used to quickly filter large data sets in order to detect potential patterns that form the basis of a more detailed analysis. The representations include global discretization, which can be efficiently analyzed using a suffix tree, local discretization with a corresponding random projection algorithm for locating similar pairs of subsequences, and a density-based detection method that operates on the original, real-valued data. In addition, a new variation of the multivariate motif discovery problem is proposed in which each pattern may span only a subset of the input features. An algorithm that can efficiently discover such “subdimensional” patterns was developed and evaluated. The discovery algorithms are evaluated by measuring the detection accuracy of discovered patterns relative to a set of expected patterns for each data set. The data sets used for evaluation are drawn from a variety of domains including speech, on-body inertial sensors, music, American Sign Language video, and GPS tracks.

ABSTRACT

We address the feature selection problem for hidden Markov models (HMMs) in sequence classification. Temporal correlation in sequences often causes difficulty in applying feature selection techniques. Inspired by segmental k-means segmentation (SKS), we propose Segmentally Boosted HMMs (SBHMMs), where the state-optimized features are constructed in a segmental and discriminative manner. The contributions are twofold. First, we introduce a novel feature selection algorithm, where the temporal dynamics are decoupled from the static learning procedure by assuming that the sequential data are piecewise independent and identically distributed. Second, we show that the SBHMM consistently improves traditional HMM recognition in various domains. The reduction of error compared to traditional HMMs ranges from 17% to 70% in American Sign Language recognition, human gait identification, lip reading, and speech recognition.

ABSTRACT

Fundamentally, journalism is aimed at collecting news information and disseminating that information with a layer of contextualization and understanding provided by journalists. Recent advances in computational technology are rapidly affecting how news information is gathered, reported and distributed. Furthermore, new avenues for aggregating, visualizing, summarizing, consuming, and collaborating on news are increasingly becoming popular and challenging traditional practices of Journalism. Following the success of text search, image and video search questions are now poised to make a bigger impact to journalism and other related fields. Computation and Journalism individually share a deep routed interest in Information, and the value it provides to society. The concept of Information Quality, the measure of the value that the information provides to the user of that information, brings these two disciplines together. In computing and information sciences, information quality is used to describe the degree of excellence in communicating knowledge or intelligence and is composed of different facets such as accuracy, reliability, comprehensiveness, currency, and validity. In journalism, where the conveyance of quality information is paramount, principles such as accuracy, fairness, thoroughness, and transparency guide journalists in communicating quality information. Traditionally, journalism has also entailed an ethos of working on the side of the citizenry to provide them with quality information they need to make informed decisions in the process of their daily lives. However, the plethora of un-vetted blogs, podcasts, videos and other online media, generated by users or by corporations with subjective biases have led to significant compromise in information quality. Collaborative knowledge generation (wikipedia), and citizen journalism, are showing new ways of how information and (global) news can be shared. However, as the Web and the Internet continue to grow and as computing technologies pervade through the planet, a thorough study of the process of journalism and the deep computational aspects of such processes need to be undertaken. To this end, the PI’s research group at Georgia Institute of Technology is interested in understanding how computational advances impact the field of journalism. The long term aim is to make novel contributions by developing computational technologies to better support the goals of journalism. To launch this effort, they are organizing a Symposium on Computation + Journalism at GA Tech, in Atlanta, GA, February 22-23, 2008. The goal of this symposium is to bring together stakeholder from the all aspects of Journalism, Media, and Computation. Participants in panels, presentations and breakout groups will discuss these issues and create a roadmap towards answering these questions that bring together computation and journalism.

My students decided to play a very nice joke on me. This morning I walked in to find my office open (and it was not!)

check out the left image, with door closed and the right image with the door open.

And, then the inside of the office was kinda different too.

• We had over 230 registered attendees. Thanks to all participants, panelists, and speakers.
• Use our Social Network (http://cj.crowdvine.com/) to continue the conversation.
• Join the FACEBOOK group (http://git.facebook.com/group.php?gid=18427444784)
• Use the tag “CnJ” on all blog posts and photo/video posts on the web, so we can collect them
• Videos of the event are now available here.

We received around 60 submissions to this track and expect a few NECTAR (new scientific and technical advances in research) submissions too (DUE Feb 18, 2008). The primary track submissions are in process of review).

Abstract Submission Deadline: January 25, 2008 *DONE*
Paper Submission Deadline: January 30, 2008 *DONE*
Author Notification Deadline: April 1, 2008