| dc.description.abstract | Image segmentation has been increasingly applied in medical settings as recent developments have skyrocketed the potential applications of deep learning. Urology, specifically, is one field of medicine that is primed for the adoption of a real-time image segmentation system with the long-term aim of automating endoscopic stone treatment. In this project, we explored supervised deep learning models to annotate kidney stones in surgical endoscopic video feeds. We describe how we built a dataset from the raw videos and how we developed a pipeline to automate as much of the process as possible. For the segmentation task, we adapted and analyzed three baseline deep learning models -- U-Net, U-Net++, and DenseNet -- to predict annotations on the frames of the endoscopic videos with the highest accuracy above 90%. After selecting a high- performing model, we performed a comparative analysis of the model's performance on digital vs. fiberoptic feeds and fragmentation vs. dusting treatments relative to expert annotation. The findings suggest substantial agreement (0.8 Cohen’s kappa) between model and expert segmentations, which is promising for widespread clinical adoption of the live system. In addition to practical analysis, we also deployed the model in a live system in real ORs during real procedures, demonstrating our proof-of-concept to guide future work. | |
