dc.contributor.author | Li, Liang | |
dc.contributor.author | Mazomenos, Evangelos | |
dc.contributor.author | Chandler, James H. | |
dc.contributor.author | Obstein, Keith L. | |
dc.contributor.author | Valdastri, Pietro | |
dc.contributor.author | Stoyanov, Danail | |
dc.contributor.author | Vasconcelos, Francisco | |
dc.date.accessioned | 2023-02-13T20:25:25Z | |
dc.date.available | 2023-02-13T20:25:25Z | |
dc.date.issued | 2022-12-14 | |
dc.identifier.issn | 1361-8415 | |
dc.identifier.other | eISSN 1361-8423 | |
dc.identifier.other | PubMed ID36549045 | |
dc.identifier.uri | http://hdl.handle.net/1803/17996 | |
dc.description.abstract | We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, handcrafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics. | en_US |
dc.description.sponsorship | This work was supported by the Wellcome/EPSRC Centre for In-terventional and Surgical Sciences (WEISS) at UCL (203145Z/16/Z) , EPSRC (EP/P027938/1) , and H2020 FET (GA863146) . Danail Stoyanov is supported by a Royal Academy of Engineering Chair in Emerging Technologies (CiET1819/2/36) and an EPSRC Early Career Research Fellowship (EP/P012841/1) . Liang Li is supported by the National Natural Science Foundation of China (62203383) and (62088101) . | en_US |
dc.language.iso | en_US | en_US |
dc.publisher | Medical Image Analysis | en_US |
dc.rights | © 2022 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). | |
dc.source.uri | https://pdf.sciencedirectassets.com/272154/1-s2.0-S1361841522X0008X/1-s2.0-S1361841522003371/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGsaCXVzLWVhc3QtMSJHMEUCIQCFnDHWAbaKryIxsasg84ibtoog0%2FZ%2FuI7CTNKcQ5IxVQIgX8wtbhZAg%2BkfhpNeIdCUKtUno34PVRUGaonQyD5xLh4q1QQI9P%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAFGgwwNTkwMDM1NDY4NjUiDKjdvh2St4i6Z7dmeSqpBFLz1nmqr35PAkW%2BycoF7Q2WdTNlVWFOoABQrBuCFge8lehOyiRfMbu17T4FXVCe6iaoyvMP4D2L5flEvtMGiW%2FBnNPIxQylTu8pNFRBPAnfk7USPzTWPulznEkOucziRXymWnyGzQFF%2BggV71sMcYzqJUmY7AggbIu6IRa0eJ6gNIjKj7pxs2V8WomMPCzhWH%2F1uv5MmV0LVJDJLxXMzguwxmy5Su%2F0A0aWFEJpZQdI%2Bd81PmKDOaS%2FpExP6eBu0dAnH%2BHvKv0T5Ceb7uPR2Q56oMaB8VfVrjVhFax43WVytd4ks%2Fj1lpHqrB3aWm8hKGKige%2BxcfNgP%2BLx%2FhXLEczx3%2F5UhioUwV9FJXrujRB5CKGaGTWwngy%2Fu6e9xmCjsvNoNBvNc6vl1nttIIPSezi%2Fd6CRyenPR7qHTCi%2FyRqY84fafO4zFItzEBUpLeXMEppZiRL552iDUTp4TbKAgf17uA3zrio1OTa1FMdj6gM8uJj05fMSVQlwWv5lZU%2BVE38sGazFKe9z%2B33VgSBizCivNwzxshIQ3gPZJhJ%2FDZsCGeHT6iSFyRQQDrQDQbcaikvcNZohMy067MG3PtLtJJ8J7trvGwQfZPWZPqx2EpnFYplA95Es%2F9RUZD7U06Sts4z6XJlspFcyw%2Fzip1%2BGfLu5kmwjcECU9tyzdduUitCVy8W1nSsteaRw%2BeeGaUKdxbNMch9JDYtVM%2FWj0pKaO3zosRlDRa4gETAwioyqnwY6qQHHs9ztrtjrqsSBot02HOym6cbVhkh%2FA1wo6bTMnNim9js3gqZzmDMhr5Y%2Fejdfe9ug3mCO04BtDFSyxvZiCfg8omYyDsrwaHwu9AUhN%2B7QYz%2FURS9j%2BxQqneQNvCXZeazJ2MwwFpUNwK7Nglk0vnuNkXITzbv4RvEfsrjvz7gkAn3OdgGxWHuiJmyG0qlo%2Fb2mpr8XcVneSAP2jKZc6h2ax6lXvSvYkaLp&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230213T201848Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTY4BQFQZ4I%2F20230213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=2f49c27a8c5b092859520ab075f70b27bce5d3d353f35cd337584c0ea8378377&hash=9f5e19b486b1781d437e1406dc1e4a22bb863bcba33fc905a7fe523f89a3b70e&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S1361841522003371&tid=spdf-a9deea19-f89d-4b06-9b39-995cbca78429&sid=a0c8e19c9a9e7140ec6b6db6d1c72610b648gxrqa&type=client&ua=131058595d5057535404&rr=7990415d4fdcf7e0&cc=us | |
dc.subject | Medical image processing | en_US |
dc.subject | Optical flow | en_US |
dc.subject | mage mosaicking | en_US |
dc.subject | Pose graph optimisation | en_US |
dc.subject | Endoscopic image mosaicking | en_US |
dc.title | Robust endoscopic image mosaicking via fusion of multimodal estimation | en_US |
dc.type | Article | en_US |
dc.identifier.doi | 10.1016/j.media.2022.102709 | |