Design and Applications of Intelligent Human-Computer Interaction Systems for Autism Spectrum Disorder intervention

Abstract

Autism Spectrum Disorder (ASD) is a common neuro-developmental disorder with core deficits in social communication. Cost and resource limitations often impede access to interventions. Since many children with ASD show affinities towards computer-based technologies, Human-Computer Interaction (HCI) systems hold promise as alternative means for providing low-cost and accessible interventions for the children. While initial studies in this area are promising, they suffered from i) limited interaction and communication between users and computers, as well as ii) labor-intensive measures of the users’ within-system behaviors. This dissertation seeks to address these limitations by designing and applying Collaborative Virtual Environments (CVEs) to facilitate realistic interaction and flexible communication between multiple users, as well as intelligent HCI systems to automatically index important aspects of their behaviors. Specifically, we have designed CVEs to encourage communication and collaboration between children with ASD and their typically developing peers. A mathematical model was developed to implement multiple collaborative strategies that required important collaborative interactions between the children. User studies with the target population demonstrated the feasibility of the CVEs. We subsequently designed intelligent HCI systems to automatically measure the children’s within-system behaviors. First, we designed an intelligent agent that could communicate and play games with the children within a CVE in order to measure both communication and collaboration skills of the children. The intelligent agent was designed with a novel hybrid method that combined data-driven and rule-based methods considering the limited available data in this area. User studies indicated its capability to communicate and play games with the children, as well as its potential to automatically measure their communication and collaboration skills. Second, we proposed a framework to measure cognitive load of the children using multimodal fusion and machine learning methods. Results indicated that the multimodal fusion methods outperformed single modality classifications for the measurements. This dissertation contributes towards designing intelligent HCI systems to automatically measure important aspects of human behaviors, and investigating the potential to address social communication deficits in the ASD population through innovative application of CVEs and intelligent HCI systems.