dc.contributor.author | Liu, Xiaoyun | |
dc.contributor.author | Esser, Daniel | |
dc.contributor.author | Wagstaff, Brandon | |
dc.contributor.author | Zavodni, Anna | |
dc.contributor.author | Matsuura, Naomi | |
dc.contributor.author | Kelly, Jonathan | |
dc.contributor.author | Diller, Eric | |
dc.date.accessioned | 2023-02-03T19:44:29Z | |
dc.date.available | 2023-02-03T19:44:29Z | |
dc.date.issued | 2022-12-07 | |
dc.identifier.citation | Liu, X., Esser, D., Wagstaff, B. et al. Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning. Sci Rep 12, 21130 (2022). https://doi.org/10.1038/s41598-022-25572-w | en_US |
dc.identifier.issn | 2045-2322 | |
dc.identifier.other | PubMed ID36476715 | |
dc.identifier.uri | http://hdl.handle.net/1803/17982 | |
dc.description.abstract | Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representative environment using ultrasound imaging. Our work proposes an attention-based hierarchical deep learning approach and adapts the success of transfer learning towards solving the multi-task tracking problem with limited dataset. To train the neural networks, we generate a representative dataset of a robotic capsule within ex-vivo porcine stomachs. Experimental results show that the accuracy of capsule state classification is 97%, and the mean estimation errors for orientation and centroid position are 2.0 degrees and 0.24 mm (1.7% of the capsule's body length) on the hold-out test set. Accurate detection of the capsule while manipulated by an external magnet in a porcine stomach and colon is also demonstrated. The results suggest our proposed method has the potential for advancing the wireless capsule-based technologies by providing accurate detection of capsule robots in clinical scenarios. | en_US |
dc.description.sponsorship | This work was supported by the Natural Sciences and Engineering Research Council of Canada through the Discovery Grant program grant no. 2014-04703. | en_US |
dc.language.iso | en_US | en_US |
dc.publisher | Scientific Reports | en_US |
dc.rights | This is an open access article distributed under the terms of the Creative Commons CC BY license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
You are not required to obtain permission to reuse this article.
To request permission for a type of use not listed, please contact Springer Nature | |
dc.source.uri | https://www.nature.com/articles/s41598-022-25572-w#Ack1 | |
dc.title | Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning | en_US |
dc.type | Article | en_US |
dc.identifier.doi | 10.1038/s41598-022-25572-w | |