Every day in the operating room, surgeons make complex visual transformations in their minds.
Any pre-operative information that they have based on medical imaging is usually static; however, during surgery there are many moving parts. That means that any surgical plan up on a screen isn’t keeping pace, and it takes a lot of mental effort to align what they see on-screen with what’s on their operating table.
Computational vision scientist Richard Wildes, Associate Director of York University’s Vision: Science to Applications (VISTA) program, wants to make their lives easier by creating tools that would update the surgical plan to match the surgery as it progresses.
It all starts with teaching computers to see in a similar manner to how we see, taking on the task of manipulating medical images like x-rays or CT scans in real time, just like a surgeon does mentally.
“[Surgeons have] the surgical plan available to them, typically on a video display someplace above where they’re working, and down in front of them, there would be the patient,” explains Wildes. “And guess what? The geometry, the layout of the organs in front of them live is very different than those captured, say, in the pre-operative imaging.
“So the surgeon has to do this very difficult, live, in-the-moment transformation between what they’re seeing on front of them on the operating table, and what they see up here in the video display. Of course, surgeons can do this, they’re very talented people; but it’s a very mentally demanding task.”
The idea is to use a video camera installed above the surgical field, giving a bird’s eye view of what is happening in the operating room. Computer vision technology would then reconstruct the 3D layout in real time and apply a transformation to the pre-operative imagery, aligning it with what the surgeon sees on the operating table.
This technology could also one day help drive surgical robotics, which are currently operated by human surgeons. Routine portions of the surgery could be assisted by computer vision to provide a surgical robot with a degree of autonomous operation, decreasing the load on human surgeons.
As computer vision continues to develop, it will play an ever greater role in assisting with complex tasks in real time. In medicine, it could help surgeons stay fully in the moment, validating the mental manipulations they do daily to help them provide the best care.
