TESS: A tool for optically measuring digital reading interactions from screen recordings

Abstract

Educational reading materials are increasingly available to students in digital environments. The use of digital education materials is likely to increase as economic trends continue to drive down the price of electronic books and the devices needed to access them. However, little is known of how electronic texts should be designed to increase learner comprehension, engagement and enjoyment. Expanding our understanding of design principles for digital reading environments will require extensive study of these environments. Currently, studies of digital reading are hampered by the available tools for monitoring user engagement. This dissertation presents TESS (Text- position Evaluation from a Screen-recorded Session), a tool that tracks and codes user actions in reading environments using screen recordings. TESS improves on previous tools by focusing on behaviors that are of interest to the reading research community, working with all stimuli, and scaling for studies with hundreds of participants. This dissertation describes how TESS works, tests to evaluate each part of the pipeline and their results, and experiments on reading practices among middle schoolers that used TESS.