AUGMENTED REALITY IN ROBOTIC SURGERY: FIRST-IN-HUMAN WITH NVIDIA HOLOSCAN AND DELTATEC VIDEO SOLUTIONS

DELTATEC, a Belgian company, leader in professional video solutions for over 30 years, and Orsi Academy, an open and inclusive ecosystem for the safe and effective implementation of new technologies in medical practice, have contributed to a premiere, with the first use of augmented reality in a robotic surgery.

Dr. Ruben De Groote and Hannes Van Den Bossche, urology surgeons at the Onze-Lieve-Vrouw (OLV) Hospital, assisted by Orsi’s Academy’s engineers Jente Simoens and Jasper Hofman, stand as the very first to leverage the NVIDIA Holoscan edge AI computing platform and DELTATEC video solutions in a real, human surgery.

In surgery or interventional oncology, increasing the precision and safety of operations requires real-time advanced technologies that can improve the surgeon's perception and highlight potential risks.

Imaging is an essential tool along the cancer treatment journey. When used before surgery, it can help diagnose cancer, locate tumors, and plan treatment, while during surgery, it can enable less invasive and more precise interventions.

However, a major problem that has hindered augmented reality for decades is the occlusion of surgical instruments during the augmented surgical field.

The Orsi Academy had already solved the problem of offline detection of instruments during augmented reality through an AI-based pipeline. However, this approach was insufficient for real-time use due to the low-latency and high computing power video streaming requirements.

Real-time augmentation

The high computational and graphical intensity of the multimodal workload made it difficult to achieve augmented reality in real-time. This has limited its potential use in live operations and therefore its usefulness for surgeons.

DELTATEC and its low-latency 12G-SDI capture and playout card, specifically developed for AI pipelines, combined with NVIDIA's Clara AGX development kit, made it possible to optimize this end-to-end pipeline, including streaming, preprocessing, AI-based computing, and rendering. In addition, the automatic failover of DELTATEC's 12G-SDI I/O card is an important feature for this type of assisted surgical augmentation as it ensures the continuity of the procedure if the augmentation is deactivated.

Seeing Through Organs

During delicate surgical interventions, no single imaging modality can provide all the useful information that a surgeon needs.

Preoperative imaging allows the surgeon to visualize the complete anatomy, but during endoscopic surgery, the surgeon can only see the operative surface. That's why multimodal imaging is essential, and augmented reality allows for the fusion of different imaging modalities to enhance the surgical scene with preoperative images.

In the case of partial nephrectomy, preoperative CT imaging and intraoperative endoscopy can be combined in augmented reality to guide the surgeon in removing the tumor and uncovering hidden vulnerable anatomy with increased precision, whether performed through open surgery or minimally invasive laparoscopy.

Industry first

Following this unique augmented intervention, here's what Dr. De Groote had to say: "By minimizing latency to an absolute minimum, augmented reality is now ready to be used in clinical practice to optimally guide surgeons during complex surgical cases."

Dr. Pieter De Backer of the Orsi Academy said: "The potential of augmented reality has been a long-standing challenge in surgery. Robotic surgery is particularly suited to push these boundaries through the integration of different technologies.”

Gérald OLIVIER, Head of Product Marketing at DELTATEC comments: “We are very proud to have been involved in this project alongside NVIDIA and Orsi. As leader in professional video solutions for over 30 years, DELTACAST has developed specific video cards adapted to the needs of AI computing pipelines. The very low latency of our 12G-SDI cards has allowed, for example, the addition of real-time AR overlays without adding any delay while its automatic failover is a true safety feature for the surgeon, preventing video signal interruption in case of power outage.”

 

Get the recording of this Success Story presentation by registering here