The objective of this research is to develop a cyber-physical system to display the inside of a patient on the skin through a 3D projector- array and a micro camera cluster, giving the appearance of “transparent skin” and enabling single incision surgery with the visual benefits of open cavity surgery.
The major difficulty in minimally invasive surgery is the loss of natural visual perception and hand-eye coordination, which results in a higher skill requirement, longer training and actual surgery time. This system will give surgeons an "X-ray" vision experience, since they see directly through the skin, and remove a spatial bottleneck and additional scarring caused by laparoscopes.
The broad challenges being addressed in this project are reducing the skill requirements to successfully perform an MIS (minimally invasive surgery); reducing the invasiveness, cost, and duration of MIS; and improving the efficiency of surgery training. The expected outcomes of this research project will be improved dexterity for MIS surgeons and significant economic growth in MIS and other healthcare-related fields with numerous benefits for the nation-at-large.
We are developing a set of micro-cameras that: occupy no space required by surgical tools, produce no additional scarring to the patient, and transfer wireless high-definition video images. Our research will create a virtual view generating system to project the panoramic 3D videos from all cameras to the right spot on the patient’s body with geometry and color distortion compensation. A surgeon-camera-interaction system is under-development to allow surgeons to control viewpoint with gesture recognition and finger tracking.
This project benefits the millions of surgeries capable of being performed through a single incision in the abdomen by providing virtually transparent skin to surgeons who will enjoy all the visual benefits of open-cavity surgery without all the associated risks to the patient. The goals of this research are extremely “hands-on” and immediately applicable to outreach activities that can excite youth, minority students, and others about the science, medicine and engineering careers.
3D Surface Reconstruction:
Jaime Sanchez, M.D. (USF Health and Tampa General
1. Multi-micro-camera video mosaicing: Data, Code, Result1, Result2.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation
Department of Computer Science and Engineering • 4202 E. Fowler Ave • Tampa, FL 33620 •
Created by: Emmanuel Stinson - Send comments to email@example.com