Models of Adaptation in Intelligent Human-Machine Interaction and Their Applications to Elder Care and Autism Spectrum Disorder Intervention

Abstract

The role of human-machine interaction (HMI) has been increasingly important in our everyday lives. This dissertation focused on creating formal methods, algorithms, and architectures for adaptive HMI with specific applications to elder care and autism spectrum disorder (ASD) intervention. Human-machine systems have been explored to engage older adults in activity-oriented therapies and provide treatments for individuals with ASD. While these systems are promising, they are limited in their ability to i) understand the implicit mental states of a user; ii) provide many-to-one interaction with multiple users; and iii) generalize design methods for a variety of interaction scenarios and learn the relationship between users’ responses and system behaviors. This research seeks to address these limitations by developing models of people for mental states estimation, developing novel socially assistive robotic (SAR) systems, and designing generalized models of interaction including a model of machine. First, a data-driven mental state models of individuals with ASD based on their electroencephalogram (EEG) responses was built. Feature engineering and machine learning methods were used to recognize four affective states and mental workload. Results demonstrated the possibility of group-level affect and workload recognition with high accuracy during realistic driving tasks. Second, a novel multi-user engagement-based SAR architecture, ROCARE, for elder care was designed. ROCARE is a user-centric model tied to the core area of engagement and featured multi-user human-robot interaction (HRI) and individualized activity management. Three closed-loop SAR systems, which provided various physical, cognitive, and social stimuli using multimodal interaction, were developed based on ROCARE for one-to-one and triadic interaction with older adults. Laboratory and field studies on these systems indicated positive acceptance and engagement of older adults, the potential for ROCARE-based systems to involve more than one older adult, to facilitate interpersonal communication, and to quantitatively measure older adults’ social interaction and activity engagement. Finally, a new mathematical model for multi-user HRI that formally defined and integrated a model of user, a model of robot, and a model of interaction was proposed. The model performed online planning during a simulated multi-user HRI task.