From Audio to Learning Analytics: Evaluating Automated Speech Processing in C2STEM Collaborative Modeling
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This thesis investigates the feasibility of using automated speech recognition and speech diarization to analyze collaborative classroom discourse in STEM learning environments. Using audio recordings from the C2STEM platform, an automated pipeline combining WhisperX transcription and Pyannote diarization was developed and evaluated against human-annotated transcripts. We measured the performance with metrics such as word error rate, n-gram overlap, semantic similarity, and diarization error rate. We then also investigate how these transcription errors impact downstream educational data-mining tasks such as transcript segmentation, collaborative problem-solving (CPS) behavior analysis, and summarization. Our results show that while word-level accuracy was relatively low, our semantic similarity remained high, and diarization achieved moderate success. Despite this, errors in transcription and diarization do seem to substantially affect fine-grained analyses such as CPS classification. These findings suggest that speech processing tools are able to capture the overall structure of collaborative discourse, but the capacity of the technology still remains limited.