Investigating the cognitive processing of experience for decision making in robots: accounting for internal states and appraisals

Abstract

Real-time search techniques have been used extensively in the areas of task planning and decision making. In order to be effective, however, these techniques require task-specific domain knowledge in the form of heuristic or utility functions. These functions can either be embedded by the programmer, or learned by the system over time. Unfortunately, many of the reinforcement learning techniques that might be used to acquire this knowledge generally demand static feature vector representations defined a priori. Current neurobiological research offers key insights into how the cognitive processing of experience may be used to alleviate dependence on pre-programmed heuristic functions as well as on static feature representations. Research also suggests that emotion-based appraisals are influenced by such processing and that these appraisals integrate with the cognitive decision-making process, providing a range of useful and adaptive control signals that focus, inform, and mediate deliberation. While the integration of emotion and cognition may limit an agent’s ability to find the most optimal solution, it is argued here that many real-world tasks only require adequate solutions, so long as those solutions can be identified quickly. This dissertation investigates how experience, stored within episodic memory, may be processed to develop a set of emotion-based appraisals that can then be used as a guide for future deliberation. These appraisals include techniques for identifying relevant information, estimating utility, predicting and adjusting for urgency, and checking fit. When derived from experience, each appraisal should contribute uniquely to deliberation and enable robotic systems to quickly determine acceptable solutions for complex tasks.