Webcam-Based Cognitive State Detection for Software Developers: System Development and Cross-Domain Analysis

Abstract

Cognitive monitoring systems increasingly assume that mental states manifest consistently across task domains, enabling universal deployment without domain-specific calibration. This research challenges this assumption through systematic cross-domain validation of webcam-based cognitive state detection across programming, mathematical problem-solving, and fatigue assessment contexts. Using a comprehensive processing pipeline built on MediaPipe Face Mesh, we extracted eye movement metrics including blink patterns, pupil dynamics, PERCLOS, and gaze characteristics from three established datasets: UTA-RLDD for fatigue detection, EMIP for programming-specific cognitive load, and Krejtz for mathematical cognitive load. Our findings reveal a fundamental dichotomy between universal fatigue detection and domain-specific cognitive load detection. Fatigue detection achieved strong cross-individual generalization with 84% accuracy using Leave-One-Subject-Out validation, demonstrating consistent physiological signatures dominated by blink-related features across different task contexts. In contrast, cognitive load detection exhibited substantial domain-specificity, with models achieving 67-76% accuracy within training domains but dropping to 22-31% accuracy when applied across domains—below chance performance, indicating systematic misclassification rather than random failure. The system achieved real-time performance suitable for practical deployment, with 12.3ms processing latency and modest computational requirements enabling integration with existing development environments. These results demonstrate that while fatigue monitoring can be universally deployed across cognitive contexts through consistent autonomic nervous system indicators, cognitive load detection requires domain-specific approaches that account for the distinct neural architectures and expertise patterns underlying different types of cognitive work. This domain-specificity challenges prevailing assumptions in cognitive monitoring re- search and establishes empirical foundations for developing more effective, context-aware cognitive support systems.