Paper in AAAI’s ICWSM (2017) “Selfie-Presentation in Everyday Life: A Large-Scale Characterization of Selfie Contexts on Instagram”

Paper

  • J. Deeb-Swihart, C. Polack, E. Gilbert, and I. Essa (2017), “Selfie-Presentation in Everyday Life: A Large-Scale Characterization of Selfie Contexts on Instagram,” in In Proceedings of The International AAAI Conference on Web and Social Media (ICWSM), 2017. [PDF] [BIBTEX]
    @InProceedings{    2017-Deeb-Swihart-SELLCSCI,
      author  = {Julia Deeb-Swihart and Christopher Polack and Eric
          Gilbert and Irfan Essa},
      booktitle  = {In Proceedings of The International AAAI Conference
          on Web and Social Media (ICWSM)},
      month    = {May},
      organization  = {AAAI},
      pdf    = {http://www.cc.gatech.edu/~irfan/p/2017-Deeb-Swihart-SELLCSCI.pdf},
      title    = {Selfie-Presentation in Everyday Life: A Large-Scale
          Characterization of Selfie Contexts on Instagram},
      year    = {2017}
    }

Abstract

Carefully managing the presentation of self via technology is a core practice on all modern social media platforms. Recently, selfies have emerged as a new, pervasive genre of identity performance. In many ways unique, selfies bring us full circle to Goffman—blending the online and offline selves together. In this paper, we take an empirical, Goffman-inspired look at the phenomenon of selfies. We report a large-scale, mixed-method analysis of the categories in which selfies appear on Instagram—an online community comprising over 400M people. Applying computer vision and network analysis techniques to 2.5M selfies, we present a typology of emergent selfie categories which represent emphasized identity statements. To the best of our knowledge, this is the first large-scale, empirical research on selfies. We conclude, contrary to common portrayals in the press, that selfies are really quite ordinary: they project identity signals such as wealth, health and physical attractiveness common to many online media, and to offline life.

Tags: , , , , | Categories: Computational Journalism, Computational Photography and Video, Computer Vision, Face and Gesture, Julia Deeb-Swihart, Papers, Social Computing | Date: May 18th, 2017 | By: Irfan Essa |

No Comments »



Paper in IJCNN (2017) “Towards Using Visual Attributes to Infer Image Sentiment Of Social Events”

Paper

  • U. Ahsan, M. D. Choudhury, and I. Essa (2017), “Towards Using Visual Attributes to Infer Image Sentiment Of Social Events,” in Proceedings of The International Joint Conference on Neural Networks, Anchorage, Alaska, US, 2017. [PDF] [BIBTEX]
    @InProceedings{    2017-Ahsan-TUVAIISSE,
      address  = {Anchorage, Alaska, US},
      author  = {Unaiza Ahsan and Munmun De Choudhury and Irfan
          Essa},
      booktitle  = {Proceedings of The International Joint Conference
          on Neural Networks},
      month    = {May},
      pdf    = {http://www.cc.gatech.edu/~irfan/p/2017-Ahsan-TUVAIISSE.pdf},
      publisher  = {International Neural Network Society},
      title    = {Towards Using Visual Attributes to Infer Image
          Sentiment Of Social Events},
      year    = {2017}
    }

Abstract

Widespread and pervasive adoption of smartphones has led to instant sharing of photographs that capture events ranging from mundane to life-altering happenings. We propose to capture sentiment information of such social event images leveraging their visual content. Our method extracts an intermediate visual representation of social event images based on the visual attributes that occur in the images going beyond
sentiment-specific attributes. We map the top predicted attributes to sentiments and extract the dominant emotion associated with a picture of a social event. Unlike recent approaches, our method generalizes to a variety of social events and even to unseen events, which are not available at training time. We demonstrate the effectiveness of our approach on a challenging social event image dataset and our method outperforms state-of-the-art approaches for classifying complex event images into sentiments.

Tags: , , , , | Categories: Computational Journalism, Computational Photography and Video, Computer Vision, Machine Learning, Papers, Unaiza Ahsan | Date: May 18th, 2017 | By: Irfan Essa |

No Comments »



Paper in IEEE WACV (2017): “Complex Event Recognition from Images with Few Training Examples”

Paper

  • U. Ahsan, C. Sun, J. Hays, and I. Essa (2017), “Complex Event Recognition from Images with Few Training Examples,” in IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. [PDF] [arXiv] [BIBTEX]
    @InProceedings{    2017-Ahsan-CERFIWTE,
      arxiv    = {https://arxiv.org/abs/1701.04769},
      author  = {Unaiza Ahsan and Chen Sun and James Hays and Irfan
          Essa},
      booktitle  = {IEEE Winter Conference on Applications of Computer
          Vision (WACV)},
      month    = {March},
      pdf    = {http://www.cc.gatech.edu/~irfan/p/2017-Ahsan-CERFIWTE.pdf},
      title    = {Complex Event Recognition from Images with Few
          Training Examples},
      year    = {2017}
    }

Abstract

We propose to leverage concept-level representations for complex event recognition in photographs given limited training examples. We introduce a novel framework to discover event concept attributes from the web and use that to extract semantic features from images and classify them into social event categories with few training examples. Discovered concepts include a variety of objects, scenes, actions and event subtypes, leading to a discriminative and compact representation for event images. Web images are obtained for each discovered event concept and we use (pre-trained) CNN features to train concept classifiers. Extensive experiments on challenging event datasets demonstrate that our proposed method outperforms several baselines using deep CNN features directly in classifying images into events with limited training examples. We also demonstrate that our method achieves the best overall accuracy on a data set with unseen event categories using a single training example.

Tags: , , , | Categories: Computational Journalism, Computational Photography and Video, Computer Vision, PAMI/ICCV/CVPR/ECCV, Papers, Unaiza Ahsan | Date: March 27th, 2017 | By: Irfan Essa |

No Comments »



Presentation at the Machine Learning Center at GA Tech on “The New Machine Learning Center at GA Tech: Plans and Aspirations”

Machine Learning at Georgia Tech Seminar Series

Speaker: Irfan Essa
Date/Time: March 1, 2017, 12n

Abstract

The Interdisciplinary Research Center (IRC) for Machine Learning at Georgia Tech (ML@GT) was established in Summer 2016 to foster research and academic activities in and around the discipline of Machine Learning. This center aims to create a community that leverages true cross-disciplinarity across all units on campus, establishes a home for the thought leaders in the area of Machine Learning, and creates programs to train the next generation of pioneers. In this talk, I will introduce the center, describe how we got here, attempt to outline the goals of this center and lay out it’s foundational, application, and educational thrusts. The primary purpose of this talk is to solicit feedback about these technical thrusts, which will be the areas we hope to focus on in the upcoming years. I will also describe, in brief, the new Ph.D. program that has been proposed and is pending approval. We will discuss upcoming events and plans for the future.

https://mediaspace.gatech.edu/media/essa/1_gfu6t21y

Tags: , | Categories: Machine Learning, Presentations | Date: March 1st, 2017 | By: Irfan Essa |

No Comments »



Paper in M2CAI (workshop MICCAI) on “Fine-tuning Deep Architectures for Surgical Tool Detection” and results of Tool Detection Challange

Paper

  • A. Zia, D. Castro, and I. Essa (2016), “Fine-tuning Deep Architectures for Surgical Tool Detection,” in Workshop and Challenges on Modeling and Monitoring of Computer Assisted Interventions (M2CAI), Held in Conjunction with International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Athens, Greece, 2016. [PDF] [WEBSITE] [BIBTEX]
    @InProceedings{    2016-Zia-FDASTD,
      address  = {Athens, Greece},
      author  = {Aneeq Zia and Daniel Castro and Irfan Essa},
      booktitle  = {Workshop and Challenges on Modeling and Monitoring
          of Computer Assisted Interventions (M2CAI), Held in
          Conjunction with International Conference on Medical
          Image Computing and Computer Assisted Intervention
          (MICCAI)},
      month    = {October},
      pdf    = {http://www.cc.gatech.edu/~irfan/p/2016-Zia-FDASTD.pdf},
      title    = {Fine-tuning Deep Architectures for Surgical Tool
          Detection},
      url    = {http://www.cc.gatech.edu/cpl/projects/deepm2cai/},
      year    = {2016}
    }

Abstract

Visualization of some of the training videos.

Understanding surgical workflow has been a key concern of the medical research community. One of the main advantages of surgical workflow detection is real-time operating room (OR) scheduling. For hospitals, each minute of OR time is important in order to reduce cost and increase patient throughput. Traditional approaches in this field generally tackle the video analysis using hand-crafted video features to facilitate the tool detection. Recently, Twinanda et al. presented a CNN architecture ’EndoNet’ which outperformed previous methods for both surgical tool detection and surgical phase detection. Given the recent success of these networks, we present a study of various architectures coupled with a submission to the M2CAI Surgical Tool Detection challenge. We achieved a top-3 result for the M2CAI competition with a mAP of 37.6.

 

Tags: , , , | Categories: Aneeq Zia, Awards, Computer Vision, Daniel Castro, Medical, MICCAI | Date: October 21st, 2016 | By: Irfan Essa |

No Comments »



Paper (ACM MM 2016) “Leveraging Contextual Cues for Generating Basketball Highlights”

Paper

  • V. Bettadapura, C. Pantofaru, and I. Essa (2016), “Leveraging Contextual Cues for Generating Basketball Highlights,” in Proceedings of ACM International Conference on Multimedia (ACM-MM), 2016. [PDF] [WEBSITE] [arXiv] [BIBTEX]
    @InProceedings{    2016-Bettadapura-LCCGBH,
      arxiv    = {http://arxiv.org/abs/1606.08955},
      author  = {Vinay Bettadapura and Caroline Pantofaru and Irfan
          Essa},
      booktitle  = {Proceedings of ACM International Conference on
          Multimedia (ACM-MM)},
      month    = {October},
      organization  = {ACM},
      pdf    = {http://www.cc.gatech.edu/~irfan/p/2016-Bettadapura-LCCGBH.pdf},
      title    = {Leveraging Contextual Cues for Generating
          Basketball Highlights},
      url    = {http://www.vbettadapura.com/highlights/basketball/index.htm},
      year    = {2016}
    }

Abstract

2016-Bettadapura-LCCGBH

Leveraging Contextual Cues for Generating Basketball Highlights

The massive growth of sports videos has resulted in a need for automatic generation of sports highlights that are comparable in quality to the hand-edited highlights produced by broadcasters such as ESPN. Unlike previous works that mostly use audio-visual cues derived from the video, we propose an approach that additionally leverages contextual cues derived from the environment that the game is being played in. The contextual cues provide information about the excitement levels in the game, which can be ranked and selected to automatically produce high-quality basketball highlights. We introduce a new dataset of 25 NCAA games along with their play-by-play stats and the ground-truth excitement data for each basket. We explore the informativeness of five different cues derived from the video and from the environment through user studies. Our experiments show that for our study participants, the highlights produced by our system are comparable to the ones produced by ESPN for the same games.

Tags: , , , | Categories: ACM MM, Caroline Pantofaru, Computational Photography and Video, Computer Vision, Papers, Sports Visualization, Vinay Bettadapura | Date: October 18th, 2016 | By: Irfan Essa |

No Comments »



Announcing the new Interdisciplinary Research Center for Machine Learning at Georgia Tech (ML@GT)

Announcement from Georgia Tech’s College of Computing about a new Interdisciplinary Research Center for Machine Learning (ML@GT) that I will be serving as the Inaugural Director for.ML@GT

Machine Learning @ Georgia Tech Based in the College of Computing, ML@GT represents all of Georgia Tech. It is tasked with pushing forward the ability for computers to learn from observations and data. As one of the fastest growing research areas in computing, machine learning spans many disciplines that use data to discover scientific principles, infer patterns, and extract meaningful knowledge.

According to School of Interactive Computing Professor Irfan Essa, inaugural director of ML@GT, machine learning (ML) has reached a new level of maturity and is now impacting all aspects of computing, engineering, science, and business. “We are in the era of aggregation, of collecting data,” said Essa. “However, machine learning is now propelling data analysis, and the whole concept of interpreting that data, toward a new era of making sense of the data, using it to make meaningful connections between information, and acting upon it in innovative ways that bring the most benefit to the most people.”

The new center begins with more than 100 affiliated faculty members from five Georgia Tech colleges and the Georgia Tech Research Institute, as well as some jointly affiliated with Emory University.

Source: Two New Interdisciplinary Research Centers Shaping Future of Computing | Georgia Tech – College of Computing

Tags: , , | Categories: In The News, Interesting, Machine Learning | Date: October 6th, 2016 | By: Irfan Essa |

No Comments »



20 years at GA Tech

September 22nd, 2016 marked 20 years of my being at GA Tech.  My team threw me a surprise party to celebrate. Here is Spherical Image of the event.  So nice of them

Lab Party for 20 – Spherical Image – RICOH THETA

Tags: , | Categories: Events, Interesting, Personal | Date: September 22nd, 2016 | By: Irfan Essa |

No Comments »



Paper in IJCARS (2016) on “Automated video-based assessment of surgical skills for training and evaluation in medical schools”

Paper

  • A. Zia, Y. Sharma, V. Bettadapura, E. L. Sarin, T. Ploetz, M. A. Clements, and I. Essa (2016), “Automated video-based assessment of surgical skills for training and evaluation in medical schools,” International Journal of Computer Assisted Radiology and Surgery, vol. 11, iss. 9, pp. 1623-1636, 2016. [WEBSITE] [DOI] [BIBTEX]
    @Article{    2016-Zia-AVASSTEMS,
      author  = {Zia, Aneeq and Sharma, Yachna and Bettadapura,
          Vinay and Sarin, Eric L and Ploetz, Thomas and
          Clements, Mark A and Essa, Irfan},
      doi    = {10.1007/s11548-016-1468-2},
      journal  = {International Journal of Computer Assisted
          Radiology and Surgery},
      month    = {September},
      number  = {9},
      pages    = {1623--1636},
      publisher  = {Springer Berlin Heidelberg},
      title    = {Automated video-based assessment of surgical skills
          for training and evaluation in medical schools},
      url    = {http://link.springer.com/article/10.1007/s11548-016-1468-2},
      volume  = {11},
      year    = {2016}
    }

Abstract

2016-Zia-AVASSTEMS

Sample frames from our video dataset

Purpose: Routine evaluation of basic surgical skills in medical schools requires considerable time and effort from supervising faculty. For each surgical trainee, a supervisor has to observe the trainees in- person. Alternatively, supervisors may use training videos, which reduces some of the logistical overhead. All these approaches, however, are still incredibly time consuming and involve human bias. In this paper, we present an automated system for surgical skills assessment by analyzing video data of surgical activities.

Method : We compare different techniques for video-based surgical skill evaluation. We use techniques that capture the motion information at a coarser granularity using symbols or words, extract motion dynamics using textural patterns in a frame kernel matrix, and analyze fine-grained motion information using frequency analysis. Results: We were successfully able to classify surgeons into different skill levels with high accuracy. Our results indicate that fine-grained analysis of motion dynamics via frequency analysis is most effective in capturing the skill relevant information in surgical videos.

Conclusion: Our evaluations show that frequency features perform better than motion texture features, which in turn perform better than symbol/word-based features. Put succinctly, skill classification accuracy is positively correlated with motion granularity as demonstrated by our results on two challenging video datasets.

Tags: , , , | Categories: Activity Recognition, Aneeq Zia, Computer Vision, Eric Sarin, Mark Clements, Medical, MICCAI, Thomas Ploetz, Vinay Bettadapura, Yachna Sharma | Date: September 2nd, 2016 | By: Irfan Essa |

No Comments »



Fall 2016 Teaching

My teaching activities for Fall 2016 areBB1162B4-4F87-480C-A850-00C54FAA0E21

Tags: , , , , | Categories: Computational Photography, Computer Vision | Date: August 10th, 2016 | By: Irfan Essa |

No Comments »