What lies beneath: Made-in-Edmonton augmented reality system lets doctors see under a patient's' skin

A new technology developed in Edmonton allows doctors to see under a patient's skin without ever wielding a scalpel.

The system, called ProjectDR, allows medical images such as CT scans to be projected directly onto a patient's body, and move in real time as the patient does.

• U of A researchers break ground on printing human-like tissues for transplant patients
• U of A program aids Indigenous and low-income students in pursuit of medical degree

The technology relies on a motion tracking system which uses infrared cameras and markers on a patient's body to accurately project the images.

The technology can show segmented images. For instance, a doctor could choose to see only the lungs or the capillaries.

"It's kind of like if you had a flashlight or something, and you could select a part of the body and show what's inside," said Ian Watts, who developed the technology with fellow University of Alberta computing science graduate Michael Feist.

"You can think of the projector illuminating that.

"If someone was standing, you could shine this projector at them and you could see their rib cage or their spine or some other internal organ like their heart or veins."

Inside, out

Watts wrote custom software which allows the images to stay perfectly aligned, so the outside image matches what's happening inside — beneath the skin.

He said creating a program which allowed the image to track and calibrate, in real time, as a patient moves, was a challenge.

Watts is now working to improve the system's automatic calibration and to add components such as depth sensors.

The work is being supervised by Pierre Boulanger, Cisco Research Chair in Healthcare Solutions and by Greg Kawchuk, a professor in the Faculty of Rehabilitation Medicine at the U of A.

It's not always best to have them poking and prodding around-Ian Watts, augmented reality researcher

Providing a new perspective on a patient's internal anatomy could have wide-ranging applications in medical practices including chiropractic care, physical therapy and diagnostics and training, said Watts.

"Even medical professionals can sometimes have trouble telling what's happening beneath the skin, because everyone is different internally and people may have had previous medical operations," he said.

He said laparoscopic surgery — operations performed through a small incision, using a camera and thin instruments — would be the perfect surgical application for the technology.

"You're inserting small tools and a camera into a small port on the body, so the patient may not be moving but the tools themselves are," said Watts.

"So you can track them, and you would have an idea where the tips of the tools are in relation to different internal structures."

Kawchuk said the next steps are testing the program's viability in a clinical setting. The researchers plan to deploy ProjectDR in a surgical simulation laboratory to test its real-life applications.

"We are also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures," he said.

Once the pilot studies are complete, the research team expects the deployment of the system in real surgical pilot studies will quickly follow, he said.