A new reality in surgery | MUSK

Marc Katz, MD, touches the heart of the man, very softly. Then he takes his index finger and his thumb, presses them against the place of interest and… he pinches.

To zoom out.

Fortunately, the cardiothoracic surgeon is not manipulating a real heart, but rather an augmented reality (AR) model of a real heart, using a holographic headset made by Microsoft and imaging software created by Virginia-based ClearView Surgical. If surgeons and tech companies have a say, it’s the future of surgery — and it’s not that far off.

Most people have heard of Virtual Reality (VR), a completely immersive experience that replaces your real environment with a simulated environment. It’s everywhere these days: movies, video games, real estate. AR is slightly different. Instead of using a simulated environment, AR adds digital elements to your real environment. She too begins to slip into everyday life. Want to see what this sofa looks like in your living room? Open an app and you can see for yourself. It’s AR.

The software uses Microsoft’s HoloLens 2, a device anyone can buy.

As Katz manipulates the organ in all directions, he resembles someone trying to walk through a darkened room – hands outstretched in front, gently feeling furniture or walls. But from Katz’s perspective, thanks to AR, what he sees is a very real heart hovering above his second-floor office.

How it works—and it’s radically simplified—ClearView Surgical engineers build exact 3D models of patients’ hearts (though it could be any organ; Katz said the heart is the one of the most complicated because it is always in motion), using information from CT scans. In turn, these patterns then allow the surgeon to manipulate the heart in any direction. Need to dismantle a valve? No problem. What’s behind that ventricle? We’ll take a look. By combining technology with medicine, surgeons like Katz will one day be able to look at — and touch — a patient’s heart without actually opening it up. The result: zero risk, tons of rewards.

Although not yet used in actual practice (Katz and ClearView Surgical seem to think we’re still at least a year away), an idea that once sounded like a Hollywood fantasy is now much closer to becoming a non- scientific fiction.

“Imagine what this technology will allow us to do,” Katz said. Take the core apart, flip it, you name it — it’s possible, he says. All without endangering the patient. Katz even thinks the technology could eventually lead to simulations of surgery, giving the surgeon the luxury of experimentation.

Dan Neuwirth, co-owner of ClearView Surgical, said despite drastic improvements in imaging technology over the years, the way doctors view images has remained largely unchanged since the 1950s. , people in the medical field looked at these images on films or computer screens. In other words, everything is 2D. Of course, technology has made it possible to simulate in three dimensions, but in the end, the image that the doctor looks at is always flat.

A computer-generated 3D model of a heart, like what Dr. Katz would see when looking through the AR headset
This image, much like a Dr. Katz might see through his helmet, allows the user to rotate it in any direction as well as remove parts to see things that might otherwise be hidden. Green represents a valve replacement device. Provided by ClearView Surgical

Neuwirth and his co-owner Daniel Salzburg knew there was a tremendous opportunity, using today’s technology, to change that. So they approached Katz and a handful of other surgeons across the country and asked for comment. It is a partnership where each party brings the necessary value. Katz provides the imaging and surgical know-how, ClearView Surgical manufactures the augmented version – so surgeons can manipulate it however they please.

“With Dr. Katz, we can tap into the mind of a super-accomplished surgeon and together we can find ways to improve patient outcomes,” Salzburg said.

“It really is a great partnership. Dr. Katz is a very forward-thinking guy in terms of technology,” Neuwirth added.

Neuwirth and Salzburg see this as the tip of the iceberg when it comes to using AR in medicine.

“Obviously it’s a big help for really complex cases like Dr. Katz works,” Neuwirth said. “It’s one end of the spectrum. But we also see it eventually helping residents with common surgeries as well.

From a medical perspective, Katz sees this as a first step in what he hopes will be a wide range of medical applications.

“Imagine being in the operating room and taking the images we have and merging them with three-dimensional images,” Katz said. “It would be like looking through someone’s body like I was wearing x-ray glasses.”

Comments are closed.