Real time Augmented Reality Registration of Stretchable Structures for Needle Navigation through the Gluteus Muscles

Virtual Reality (VR) and Augmented Reality (AR) are two popular platforms to enhance human vision by providing additional layers of information not visible to the naked eye. The gaming industry is blooming using this technology, and the tech giants have recently manufactured high accuracy VR/AR glasses.

In this proposal, we claim that we can use the same hardware but add improved AI algorithms of alignment between stretchable domains and project pre-scanned CT images on top of the soft tissues. The current alignment done in a virtual scenario is not sufficient for surgical procedures, especially where there is a movement of the skin and fat tissues between the CT scans and the 3D positioning of the body during the medical procedure. While brain surgery navigation is accurate enough for computer assistance, it is built for hard tissues (bones) and requires heavy preparation per patient. Here we focus on a quick solution for soft tissues with minimal resource requirements.

Using state-of-the-art rigid and non-rigid geometric deep learning alignment algorithms developed recently in Dr. Raviv’s lab and the medical expertise from Dr. Tejman-Yarden’s lab, we believe we can solve a fundamental medical task. Using superior algorithms, we plan to provide a robust real-time data fusion tool of soft tissues for surgical needs. We plan to focus first on needle insertion through the Gluteus Muscles, but the same approach can be used for multiple organs and multiple treatments.