Real-time Gesture Imitation in a Soft-arm Control Robot

Abstract

In this thesis a system is developed whereby ISAC, a soft-arm control humanoid robot, can observe, track, and imitate hand motions made by a human being. This is accomplished by making use of the OpenCV libraries for Haar object detection and pre-trained Haar classifiers to detect the human’s face and hand, applying stereo vision geometry to identify the relative locations of the face and hand and to map those coordinates onto the workspace of ISAC, and by transmitting those coordinates via UDP to the arm controller, which interpolates and activates the corresponding arm motions. Thus, ISAC can imitate motions in real-time. These motions are also stored in a database on the arm control computer for later use.