Abdurrahim Yilmaz, Serra Atilla Aydin, Deniz Temur
et al.
Mohs micrographic surgery (MMS) is the gold standard technique for removing high risk nonmelanoma skin cancer however, intraoperative histopathological examination demands significant time, effort, and professionality. The objective of this study is to develop a deep learning model to detect basal cell carcinoma (BCC) and artifacts on Mohs slides. A total of 731 Mohs slides from 51 patients with BCCs were used in this study, with 91 containing tumor and 640 without tumor which was defined as non-tumor. The dataset was employed to train U-Net based models that segment tumor and non-tumor regions on the slides. The segmented patches were classified as tumor, or non-tumor to produce predictions for whole slide images (WSIs). For the segmentation phase, the deep learning model success was measured using a Dice score with 0.70 and 0.67 value, area under the curve (AUC) score with 0.98 and 0.96 for tumor and non-tumor, respectively. For the tumor classification, an AUC of 0.98 for patch-based detection, and AUC of 0.91 for slide-based detection was obtained on the test dataset. We present an AI system that can detect tumors and non-tumors in Mohs slides with high success. Deep learning can aid Mohs surgeons and dermatopathologists in making more accurate decisions.
Current orthopedic robotic systems largely focus on navigation, aiding surgeons in positioning a guiding tube but still requiring manual drilling and screw placement. The automation of this task not only demands high precision and safety due to the intricate physical interactions between the surgical tool and bone but also poses significant risks when executed without adequate human oversight. As it involves continuous physical interaction, the robot should collaborate with the surgeon, understand the human intent, and always include the surgeon in the loop. To achieve this, this paper proposes a new cognitive human-robot collaboration framework, including the intuitive AR-haptic human-robot interface, the visual-attention-based surgeon model, and the shared interaction control scheme for the robot. User studies on a robotic platform for orthopedic surgery are presented to illustrate the performance of the proposed method. The results demonstrate that the proposed human-robot collaboration framework outperforms full robot and full human control in terms of safety and ergonomics.
Due to the high error rate of a qubit, detecting and correcting errors on it is essential for fault-tolerant quantum computing (FTQC). Among several FTQC techniques, lattice surgery (LS) using surface code (SC) is currently promising. To demonstrate practical quantum advantage as early as possible, it is indispensable to propose a high-performance and low-overhead FTQC architecture specialized for a given FTQC scheme based on detailed analysis. In this study, we first categorize the factors, or hazards, that degrade LS-based FTQC performance and propose a performance evaluation methodology to decompose the impact of each hazard, inspired by the CPI stack. We propose the Bypass architecture based on the bottleneck analysis using the proposed evaluation methodology. The proposed Bypass architecture is a 2.5-dimensional architecture consisting of dense and sparse qubit layers and successfully eliminates the bottleneck to achieve high-performance and scalable LS-based FTQC. We evaluate the proposed architecture with a circuit-level stabilizer simulator and a cycle-accurate LS simulator with practical quantum phase estimation problems. The results show that the Bypass architecture improves the fidelity of FTQC and achieves both a 1.73x speedup and a 17% reduction in classical/quantum hardware resources over a conventional 2D architecture.
In response to the growing demand for precise and affordable solutions for Image-Guided Spine Surgery (IGSS), this paper presents a comprehensive development of a Robot-Assisted and Navigation-Guided IGSS System. The endeavor involves integrating cutting-edge technologies to attain the required surgical precision and limit user radiation exposure, thereby addressing the limitations of manual surgical methods. We propose an IGSS workflow and system architecture employing a hybrid-layered approach, combining modular and integrated system architectures in distinctive layers to develop an affordable system for seamless integration, scalability, and reconfigurability. We developed and integrated the system and extensively tested it on phantoms and cadavers. The proposed system's accuracy using navigation guidance is 1.020 mm, and robot assistance is 1.11 mm on phantoms. Observing a similar performance in cadaveric validation where 84% of screw placements were grade A, 10% were grade B using navigation guidance, 90% were grade A, and 10% were grade B using robot assistance as per the Gertzbein-Robbins scale, proving its efficacy for an IGSS. The evaluated performance is adequate for an IGSS and at par with the existing systems in literature and those commercially available. The user radiation is lower than in the literature, given that the system requires only an average of 3 C-Arm images per pedicle screw placement and verification
Deep Neural Networks (DNNs) based semantic segmentation of the robotic instruments and tissues can enhance the precision of surgical activities in robot-assisted surgery. However, in biological learning, DNNs cannot learn incremental tasks over time and exhibit catastrophic forgetting, which refers to the sharp decline in performance on previously learned tasks after learning a new one. Specifically, when data scarcity is the issue, the model shows a rapid drop in performance on previously learned instruments after learning new data with new instruments. The problem becomes worse when it limits releasing the dataset of the old instruments for the old model due to privacy concerns and the unavailability of the data for the new or updated version of the instruments for the continual learning model. For this purpose, we develop a privacy-preserving synthetic continual semantic segmentation framework by blending and harmonizing (i) open-source old instruments foreground to the synthesized background without revealing real patient data in public and (ii) new instruments foreground to extensively augmented real background. To boost the balanced logit distillation from the old model to the continual learning model, we design overlapping class-aware temperature normalization (CAT) by controlling model learning utility. We also introduce multi-scale shifted-feature distillation (SD) to maintain long and short-range spatial relationships among the semantic objects where conventional short-range spatial features with limited information reduce the power of feature distillation. We demonstrate the effectiveness of our framework on the EndoVis 2017 and 2018 instrument segmentation dataset with a generalized continual learning setting. Code is available at~\url{https://github.com/XuMengyaAmy/Synthetic_CAT_SD}.
In the realm of automated robotic surgery and computer-assisted interventions, understanding robotic surgical activities stands paramount. Existing algorithms dedicated to surgical activity recognition predominantly cater to pre-defined closed-set paradigms, ignoring the challenges of real-world open-set scenarios. Such algorithms often falter in the presence of test samples originating from classes unseen during training phases. To tackle this problem, we introduce an innovative Open-Set Surgical Activity Recognition (OSSAR) framework. Our solution leverages the hyperspherical reciprocal point strategy to enhance the distinction between known and unknown classes in the feature space. Additionally, we address the issue of over-confidence in the closed set by refining model calibration, avoiding misclassification of unknown classes as known ones. To support our assertions, we establish an open-set surgical activity benchmark utilizing the public JIGSAWS dataset. Besides, we also collect a novel dataset on endoscopic submucosal dissection for surgical activity tasks. Extensive comparisons and ablation experiments on these datasets demonstrate the significant outperformance of our method over existing state-of-the-art approaches. Our proposed solution can effectively address the challenges of real-world surgical scenarios. Our code is publicly accessible at https://github.com/longbai1006/OSSAR.
Paul Maria Scheikl, Balázs Gyenes, Rayan Younis
et al.
Recent advances in reinforcement learning (RL) have increased the promise of introducing cognitive assistance and automation to robot-assisted laparoscopic surgery (RALS). However, progress in algorithms and methods depends on the availability of standardized learning environments that represent skills relevant to RALS. We present LapGym, a framework for building RL environments for RALS that models the challenges posed by surgical tasks, and sofa_env, a diverse suite of 12 environments. Motivated by surgical training, these environments are organized into 4 tracks: Spatial Reasoning, Deformable Object Manipulation & Grasping, Dissection, and Thread Manipulation. Each environment is highly parametrizable for increasing difficulty, resulting in a high performance ceiling for new algorithms. We use Proximal Policy Optimization (PPO) to establish a baseline for model-free RL algorithms, investigating the effect of several environment parameters on task difficulty. Finally, we show that many environments and parameter configurations reflect well-known, open problems in RL research, allowing researchers to continue exploring these fundamental problems in a surgical context. We aim to provide a challenging, standard environment suite for further development of RL for RALS, ultimately helping to realize the full potential of cognitive surgical robotics. LapGym is publicly accessible through GitHub (https://github.com/ScheiklP/lap_gym).
Andreia F. Silva, Francisco Pimenta, Manuel A. Alves
et al.
This work discusses the rheology of several vitreous humour (VH) substitutes used in eye surgery (perfluorocarbons and silicone oils) and their flow behaviour when subjected to saccadic eye movements. Shear rheology experiments revealed that all fluids tested exhibit a constant shear viscosity, while extensional rheological experiments showed that Siluron 2000 is the only fluid tested that exhibits a measurable elasticity. To characterise the dynamics during saccadic eye movements, numerical simulations of all the VH substitutes under study were performed with the open source software OpenFOAM and compared with Vitreous Humour flow dynamics to assess their potential mechanical performance. Minor differences were found between the numerical results of a viscoelastic fluid reproducing the rheology of Siluron 2000 and a Newtonian model. Perfluorocarbon (PFLC) shows a distinct flow behaviour relative to Silicone Oils (SiO). None of the pharmacological fluids tested can adequately mimic the rheological and consequently the flow behaviour of VH gel phase (Silva et al., 2020).
Soorena Salari, Amirhossein Rasoulian, Hassan Rivaz
et al.
Early surgical treatment of brain tumors is crucial in reducing patient mortality rates. However, brain tissue deformation (called brain shift) occurs during the surgery, rendering pre-operative images invalid. As a cost-effective and portable tool, intra-operative ultrasound (iUS) can track brain shift, and accurate MRI-iUS registration techniques can update pre-surgical plans and facilitate the interpretation of iUS. This can boost surgical safety and outcomes by maximizing tumor removal while avoiding eloquent regions. However, manual assessment of MRI-iUS registration results in real-time is difficult and prone to errors due to the 3D nature of the data. Automatic algorithms that can quantify the quality of inter-modal medical image registration outcomes can be highly beneficial. Therefore, we propose a novel deep-learning (DL) based framework with the Swin UNETR to automatically assess 3D-patch-wise dense error maps for MRI-iUS registration in iUS-guided brain tumor resection and show its performance with real clinical data for the first time.
Robot-assisted surgery has made significant progress, with instrument segmentation being a critical factor in surgical intervention quality. It serves as the building block to facilitate surgical robot navigation and surgical education for the next generation of operating intelligence. Although existing methods have achieved accurate instrument segmentation results, they simultaneously generate segmentation masks for all instruments, without the capability to specify a target object and allow an interactive experience. This work explores a new task of Referring Surgical Video Instrument Segmentation (RSVIS), which aims to automatically identify and segment the corresponding surgical instruments based on the given language expression. To achieve this, we devise a novel Video-Instrument Synergistic Network (VIS-Net) to learn both video-level and instrument-level knowledge to boost performance, while previous work only used video-level information. Meanwhile, we design a Graph-based Relation-aware Module (GRM) to model the correlation between multi-modal information (i.e., textual description and video frame) to facilitate the extraction of instrument-level information. We are also the first to produce two RSVIS datasets to promote related research. Our method is verified on these datasets, and experimental results exhibit that the VIS-Net can significantly outperform existing state-of-the-art referring segmentation methods. Our code and our datasets will be released upon the publication of this work.
Aoife McDonald-Bowyer, Solène Dietsch, Emmanouil Dimitrakakis
et al.
In robotic-assisted partial nephrectomy, surgeons remove a part of a kidney often due to the presence of a mass. A drop-in ultrasound probe paired to a surgical robot is deployed to execute multiple swipes over the kidney surface to localise the mass and define the margins of resection. This sub-task is challenging and must be performed by a highly skilled surgeon. Automating this sub-task may reduce cognitive load for the surgeon and improve patient outcomes. The overall goal of this work is to autonomously move the ultrasound probe on the surface of the kidney taking advantage of the use of the Pneumatically Attachable Flexible (PAF) rail system, a soft robotic device used for organ scanning and repositioning. First, we integrate a shape-sensing optical fibre into the PAF rail system to evaluate the curvature of target organs in robotic-assisted laparoscopic surgery. Then, we investigate the impact of the stiffness of the material of the PAF rail on the curvature sensing accuracy, considering that soft targets are present in the surgical field. Finally, we use shape sensing to plan the trajectory of the da Vinci surgical robot paired with a drop-in ultrasound probe and autonomously generate an Ultrasound scan of a kidney phantom.
Zih-Yun Chiu, Albert Z Liao, Florian Richter
et al.
Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the inconsistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery (e.g., blood). Therefore in this work, we present a robust tracking approach for estimating the 6D pose of a suture needle when using inconsistent detections. We define observation models based on suture needles' geometry that captures the uncertainty of the detections and fuse them temporally in a probabilistic fashion. In our experiments, we compare different permutations of the observation models in the suture needle localization task to show their effectiveness. Our proposed method outperforms previous approaches in localizing a suture needle. We also demonstrate the proposed tracking method in an autonomous suture needle regrasping task and ex vivo environments.
We investigate the potential of machine learning models for the prediction of visual improvement after macular hole surgery from preoperative data (retinal images and clinical features). Collecting our own data for the task, we end up with only 121 total samples, putting our work in the very limited data regime. We explore a variety of deep learning methods for limited data to train deep computer vision models, finding that all tested deep vision models are outperformed by a simple regression model on the clinical features. We believe this is compelling evidence of the extreme difficulty of using deep learning on very limited data.
Yangming Li, Neeraja Konuthula, Ian M. Humphreys
et al.
Abstract. Purpose: This paper presents a scheme for generating virtual intraoperative CT scans in order to improve surgical completeness in Endoscopic Sinus Surgeries (ESS). Approach: The work presents three methods, the tip motion-based, the tip trajectory-based, and the instrument based, along with non-parametric smoothing and Gaussian Process Regression, for virtual intraoperative CT generation. Results: The proposed methods studied and compared on ESS performed on cadavers. Surgical results show all three methods improve the Dice Similarity Coefficients > 86%, with F-score > 92% and precision > 89.91%. The tip trajectory-based method was found to have best performance and reached 96.87% precision in surgical completeness evaluation. Conclusions: This work demonstrated that virtual intraoperative CT scans improves the consistency between the actual surgical scene and the reference model, and improves surgical completeness in ESS. Comparing with actual intraoperative CT scans, the proposed scheme has no impact on existing surgical protocols, does not require extra hardware other than the one is already available in most ESS overcome the high costs, the repeated radiation, and the elongated anesthesia caused by actual intraoperative CTs, and is practical in ESS.
Following the classification of genus one fibered knots in lens spaces by Baker, we determine hyperbolic genus one fibered knots in lens spaces on whose all integral Dehn surgeries yield closed 3-manifolds with left-orderable fundamental groups.
By using grid homology theory, we give an explicit algorithm for computing Ozsváth-Stipsicz-Szabó's $Υ$-invariant and the $d$-invariant of Dehn surgeries along knots in $S^3$. As its application, we compute the two invariants for all prime knots with up to 11 crossings.
We present the Round Handle Problem, proposed by Freedman and Krushkal. It asks whether a collection of links, which contains the Generalised Borromean Rings, are slice in a 4-manifold R constructed from adding round handles to the four ball. A negative answer would contradict the union of the surgery conjecture and the s-cobordism conjecture for 4-manifolds with free fundamental group.
Conjecturally, there are only finitely many Heegaard Floer L-space knots in $S^3$ of a given genus. We examine this conjecture for twist families of knots $\{K_n\}$ obtained by twisting a knot $K$ in $S^3$ along an unknot $c$ in terms of the linking number $ω$ between $K$ and $c$. We establish the conjecture in case of $|ω| \neq 1$, prove that $\{K_n\}$ contains at most three L-space knots if $ω= 0$, and address the case where $|ω| = 1$ under an additional hypothesis about Seifert surgeries. To that end, we characterize a twisting circle $c$ for which $\{ (K_n, r_n) \}$ contains at least ten Seifert surgeries. We also pose a few questions about the nature of twist families of L-space knots, their expressions as closures of positive (or negative) braids, and their wrapping about the twisting circle.
An L-space link is a link in $S^3$ on which all sufficiently large integral surgeries are L-spaces. We prove that for m, n relatively prime, the r-component cable link $K_{rm,rn}$ is an L-space link if and only if K is an L-space knot and $n/m \geq 2g(K)-1$. We also compute HFL-minus and HFL-hat of an L-space cable link in terms of its Alexander polynomial. As an application, we confirm a conjecture of Licata regarding the structure of HFL-hat for (n,n) torus links.