Machine learning based autism spectrum disorder detection from videos

Chongruo Wu, Sidrah Liaqat, Halil Helvaci, Sen Ching Samson Chcung, Chen Nee Chuah, Sally Ozonoff, Gregory Young

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Early diagnosis of Autism Spectrum Disorder (ASD) is crucial for best outcomes to interventions. In this paper, we present a machine learning (ML) approach to ASD diagnosis based on identifying specific behaviors from videos of infants of ages 6 through 36 months. The behaviors of interest include directed gaze towards faces or objects of interest, positive affect, and vocalization. The dataset consists of 2000 videos of 3-minute duration with these behaviors manually coded by expert raters. Moreover, the dataset has statistical features including duration and frequency of the above mentioned behaviors in the video collection as well as independent ASD diagnosis by clinicians. We tackle the ML problem in a two-stage approach. Firstly, we develop deep learning models for automatic identification of clinically relevant behaviors exhibited by infants in a one-on-one interaction setting with parents or expert clinicians. We report baseline results of behavior classification using two methods: (1) image based model (2) facial behavior features based model. We achieve 70% accuracy for smile, 68% accuracy for look face, 67% for look object and 53% accuracy for vocalization. Secondly, we focus on ASD diagnosis prediction by applying a feature selection process to identify the most significant statistical behavioral features and a over and under sampling process to mitigate the class imbalance, followed by developing a baseline ML classifier to achieve an accuracy of 82% for ASD diagnosis.

Original languageEnglish (US)
Title of host publication2020 IEEE International Conference on E-Health Networking, Application and Services, HEALTHCOM 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728162676
DOIs
StatePublished - Mar 1 2021
Externally publishedYes
Event22nd IEEE International Conference on E-Health Networking, Application and Services, HEALTHCOM 2020 - Shenzhen, China
Duration: Mar 1 2021Mar 2 2021

Publication series

Name2020 IEEE International Conference on E-Health Networking, Application and Services, HEALTHCOM 2020

Conference

Conference22nd IEEE International Conference on E-Health Networking, Application and Services, HEALTHCOM 2020
Country/TerritoryChina
CityShenzhen
Period3/1/213/2/21

Keywords

  • Autism Spectrum Disorder
  • Facial Keypoint Detection
  • Human Behavior Detection
  • Machine Learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems and Management
  • Health Informatics
  • Health(social science)

Fingerprint

Dive into the research topics of 'Machine learning based autism spectrum disorder detection from videos'. Together they form a unique fingerprint.

Cite this