prof.irfanessa.com

Paper

• A. Zia, Y. Sharma, V. Bettadapura, E.Sarin, and I. Essa (2017), “Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment,” in Proceedings of Information Processing in Computer-Assisted Interventions (IPCAI), 2017. [PDF] [BIBTEX]

  @InProceedings{    2017-Zia-VAMAASSA,
    author  = {A. Zia and Y. Sharma and V. Bettadapura and E.Sarin
        and I. Essa},
    booktitle  = {Proceedings of Information Processing in
        Computer-Assisted Interventions (IPCAI)},
    month    = {June},
    pdf    = {http://www.cc.gatech.edu/~irfan/p/2017-Zia-VAMAASSA.pdf},
    title    = {Video and Accelerometer-Based Motion Analysis for
        Automated Surgical Skills Assessment},
    year    = {2017}
  }

Abstract

Purpose: Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated
assessment of OSATS based surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data).
Methods: We conduct the largest study, to the best of our knowledge, for basic surgical skills assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce “entropy based” features – Approximate Entropy (ApEn) and Cross-Approximate Entropy (XApEn), which quantify the amount of predictability and regularity of fluctuations in time-series data. The
proposed features are compared to existing methods of Sequential Motion Texture (SMT), Discrete Cosine Transform (DCT) and Discrete Fourier Transform (DFT), for surgical skills assessment.
Results: We report average performance of different features across all applicable OSATS criteria for suturing and knot tying tasks. Our analysis shows that the proposed entropy based features out-perform previous state-of-the-art methods using video data. For accelerometer data, our method performs better for suturing only. We also show that fusion of video and acceleration features can improve overall performance with the proposed entropy features achieving highest accuracy.
Conclusions: Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.

• Presented at The 8th International Conference on Information Processing in Computer-Assisted Interventions, in Barcelona, SPAIN, June 20-21, 2017.
• Aneeq Zia awarded the “Young Investigator Travel Award” given to young investigators (including Ph.D. and MSc students and junior researchers) with accepted papers at IPCAI conference to attend IPCAI/CARS 2017.
• This paper was also 1 of the 12 papers voted by the audience for a 25 minute long oral presentation and discussion session on the last day of conference (based on 5 minute short presentations given by all authors on the first day).

Paper

• A. Zia, Y. Sharma, V. Bettadapura, E. L. Sarin, T. Ploetz, M. A. Clements, and I. Essa (2016), “Automated video-based assessment of surgical skills for training and evaluation in medical schools,” International Journal of Computer Assisted Radiology and Surgery, vol. 11, iss. 9, pp. 1623-1636, 2016. [WEBSITE] [DOI] [BIBTEX]

  @Article{    2016-Zia-AVASSTEMS,
    author  = {Zia, Aneeq and Sharma, Yachna and Bettadapura,
        Vinay and Sarin, Eric L and Ploetz, Thomas and
        Clements, Mark A and Essa, Irfan},
    doi    = {10.1007/s11548-016-1468-2},
    journal  = {International Journal of Computer Assisted
        Radiology and Surgery},
    month    = {September},
    number  = {9},
    pages    = {1623--1636},
    publisher  = {Springer Berlin Heidelberg},
    title    = {Automated video-based assessment of surgical skills
        for training and evaluation in medical schools},
    url    = {http://link.springer.com/article/10.1007/s11548-016-1468-2},
    volume  = {11},
    year    = {2016}
  }

Abstract

Sample frames from our video dataset

Purpose: Routine evaluation of basic surgical skills in medical schools requires considerable time and effort from supervising faculty. For each surgical trainee, a supervisor has to observe the trainees in- person. Alternatively, supervisors may use training videos, which reduces some of the logistical overhead. All these approaches, however, are still incredibly time consuming and involve human bias. In this paper, we present an automated system for surgical skills assessment by analyzing video data of surgical activities.

Method : We compare different techniques for video-based surgical skill evaluation. We use techniques that capture the motion information at a coarser granularity using symbols or words, extract motion dynamics using textural patterns in a frame kernel matrix, and analyze fine-grained motion information using frequency analysis. Results: We were successfully able to classify surgeons into different skill levels with high accuracy. Our results indicate that fine-grained analysis of motion dynamics via frequency analysis is most effective in capturing the skill relevant information in surgical videos.

Conclusion: Our evaluations show that frequency features perform better than motion texture features, which in turn perform better than symbol/word-based features. Put succinctly, skill classification accuracy is positively correlated with motion granularity as demonstrated by our results on two challenging video datasets.

Paper

• A. Zia, Y. Sharma, V. Bettadapura, E. Sarin, M. Clements, and I. Essa (2015), “Automated Assessment of Surgical Skills Using Frequency Analysis,” in International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), 2015. [PDF] [BIBTEX]

  @InProceedings{    2015-Zia-AASSUFA,
    author  = {A. Zia and Y. Sharma and V. Bettadapura and E.
        Sarin and M. Clements and I. Essa},
    booktitle  = {International Conference on Medical Image Computing
        and Computer Assisted Interventions (MICCAI)},
    month    = {October},
    pdf    = {http://www.cc.gatech.edu/~irfan/p/2015-Zia-AASSUFA.pdf},
    title    = {Automated Assessment of Surgical Skills Using
        Frequency Analysis},
    year    = {2015}
  }

Abstract

We present an automated framework for a visual assessment of the expertise level of surgeons using the OSATS (Objective Structured Assessment of Technical Skills) criteria. Video analysis technique for extracting motion quality via  frequency coefficients is introduced. The framework is tested in a case study that involved analysis of videos of medical students with different expertise levels performing basic surgical tasks in a surgical training lab setting. We demonstrate that transforming the sequential time data into frequency components effectively extracts the useful information differentiating between different skill levels of the surgeons. The results show significant performance improvements using DFT and DCT coefficients over known state-of-the-art techniques.

• Presented at 18th International Conference on Medical Image Computing and Computer Assisted Intervention, will be held from October 5th to 9th, 2015 in Munich, Germany.
• More details at the Project Website

Participated in the Dagstuhl Workshop on “Modeling and Simulation of Sport Games, Sport Movements, and Adaptations to Training” at the Dagstuhl Castle, September 13  – 16, 2015.

Motivation

Computational modeling and simulation are essential to analyze human motion and interaction in sports science. Applications range from game analysis, issues in training science like training load-adaptation relationship, motor control & learning, to biomechanical analysis. The motivation of this seminar is to enable an interdisciplinary exchange between sports and computer scientists to advance modeling and simulation technologies in selected fields of applications: sport games, sport movements and adaptations to training. In addition, contributions to the epistemic basics of modeling and simulation are welcome.

Source: Schloss Dagstuhl : Seminar Homepage

Past Seminars on this topic include

• 13272: “Computer Science in High Performance Sport – Applications and Implications for Professional Coaching” (2013)
• 11271: “Computer Science in Sport – Special emphasis: Football” (2011)
• 08372: “Computer Science in Sport – Mission and Methods” (2008)
• 06381: “Computer Science in Sport “ (2006)

Paper

• E. Thomaz, I. Essa, and G. D. Abowd (2015), “A Practical Approach for Recognizing Eating Moments with Wrist-Mounted Inertial Sensing,” in Proceedings of ACM International Conference on Ubiquitous Computing (UBICOMP), 2015. [PDF] [BIBTEX]

  @InProceedings{    2015-Thomaz-PAREMWWIS,
    author  = {Edison Thomaz and Irfan Essa and Gregory D. Abowd},
    booktitle  = {Proceedings of ACM International Conference on
        Ubiquitous Computing (UBICOMP)},
    month    = {September},
    pdf    = {http://www.cc.gatech.edu/~irfan/p/2015-Thomaz-PAREMWWIS.pdf},
    title    = {A Practical Approach for Recognizing Eating Moments
        with Wrist-Mounted Inertial Sensing},
    year    = {2015}
  }

Abstract

Recognizing when eating activities take place is one of the key challenges in automated food intake monitoring. Despite progress over the years, most proposed approaches have been largely impractical for everyday usage, requiring multiple onbody sensors or specialized devices such as neck collars for swallow detection. In this paper, we describe the implementation and evaluation of an approach for inferring eating moments based on 3-axis accelerometry collected with a popular off-the-shelf smartwatch. Trained with data collected in a semi-controlled laboratory setting with 20 subjects, our system recognized eating moments in two free-living condition studies (7 participants, 1 day; 1 participant, 31 days), with Fscores of 76.1% (66.7% Precision, 88.8% Recall), and 71.3% (65.2% Precision, 78.6% Recall). This work represents a contribution towards the implementation of a practical, automated system for everyday food intake monitoring, with applicability in areas ranging from health research and food journaling.

• Presented at the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015) held at Grand Front Osaka in Umeda, Osaka, Japan from Sep. 7-11, colocated with ISWC 2015.
• More details from Edison Thomaz’s website