RESEARCH

Virtual Reality (VR) and Augmented Reality (AR) are two popular platforms to enhance human vision by providing additional layers of information not visible to the naked eye. The gaming industry is blooming using this technology, and the tech giants have recently manufactured high accuracy VR/AR glasses.

In this proposal, we claim that we can use the same hardware but add improved AI algorithms of alignment between stretchable domains and project pre-scanned CT images on top of the soft tissues. The current alignment done in a virtual scenario is not sufficient for surgical procedures, especially where there is a movement of the skin and fat tissues between the CT scans and the 3D positioning of the body during the medical procedure. While brain surgery navigation is accurate enough for computer assistance, it is built for hard tissues (bones) and requires heavy preparation per patient. Here we focus on a quick solution for soft tissues with minimal resource requirements.

Using state-of-the-art rigid and non-rigid geometric deep learning alignment algorithms developed recently in Dr. Raviv’s lab and the medical expertise from Dr. Tejman-Yarden’s lab, we believe we can solve a fundamental medical task. Using superior algorithms, we plan to provide a robust real-time data fusion tool of soft tissues for surgical needs. We plan to focus first on needle insertion through the Gluteus Muscles, but the same approach can be used for multiple organs and multiple treatments.

Extreme precipitation events are becoming more frequent and in areas where previously never existed. Accurate/prior prediction of light, medium and intense rain events is important on a time-scales of hours to allow adequate warnings and mitigation planning, especially in remote and developing countries. Indeed, such predictions are available (albeit not optimal yet) by numerical weather prediction (NWP) models or by ground-based radar systems.

However, NWP models are very costly in terms of processing time, memory, and expert-knowledge needs, while ground-based radars are not widespread enough, especially not in developing countries or in remote regions (e.g. desert areas or over the ocean). 

Here, we propose to utilize available near-real-time wide-coverage high temporal resolution geosynchronous satellite imagery and sparse precipitation data to construct a machine learning model for predicting rain intensity classes for short-term (0-6 hours) applications. We propose a tailor-made machine-learning architecture that combines both spatial (e.g. CNN) and time-series (ESN, LSTM) machine-learning approaches that will allow the ingestion and fusion of satellite-data on different times and spatial scales.

We are looking to apply our expertise in cloud detection and classification and retrieval approaches, with the support from the TAD center experts to derive a fast and simple yet useful enough rain predictor that can be used globally and can be easily embedded and implemented on a publicly available web-platform.