From VR to 3D

An integral part of the project is the virtual and augmented reality system designed by Cattin and his team. Used for planning and monitoring surgical interventions, the VR tool can analyse a defective bone, or the part of a bone damaged by a tumour. “We’ve also trained the system to calculate the dimensions of the replacement part so that it fits nicely into the affected bone’s shape, be it a skull or another bone,” Cattin explains.

To this end, the researchers first had to develop an artificial intelligence system capable of understanding human anatomy. The innovative system’s task is to assess where tissue is missing and calculate how this section would look in a healthy bone of the same size. “Today, highly paid specialists use CAD to draw the shapes for these kinds of implants,” Cattin says.

His group has now already linked their automated planning system with the manufacture of implants, the part of the project conducted by the research team led by Florian Thieringer, senior physician for oral and cranio-maxillofacial surgery at the University Hospital Basel and head of the hospital’s in-house 3D printing lab. Once the artificial intelligence has calculated an implant’s size, the result is sent to the 3D printer in Thieringer’s lab, which then fabricates a made-to-measure implant.

Right now, the file created in the planning tool must first be saved and then restarted in the printer tool. “But our goal is starting the 3D printer by just pressing a button in the VR system,” Cattin says. And although the system works perfectly from a technical point of view, the automation is currently not in use at the hospital, as regulatory issues are yet to be settled.

Coordinated lasers

Advances in other areas are also slowly but surely being integrated into the overall project, including the most important feature: a tiny laser beam capable of making clean and precise incisions in a damaged bone. To avoid causing inadvertent harm to the surrounding area, the laser must also have access to information about where exactly in the body it’s operating and what kind of tissue it will be cutting.

To realise the intricate tool, the researchers first developed three different laser systems: one to measure how deep an incision should be, one to characterise tissue, and one to cut. “At the beginning, every PhD student was working separately on their piece of the puzzle,” Cattin explains. “Only after the individual parts worked could we concentrate on integrating them.” Which they now have done: in a published study, the MIRACLE team demonstrated how the laser types interact. They made ultra-precise incisions and, at the same time, minimised damage to the bone marrow thanks to the tool’s measuring and characterising capacities.

An air bed for the robot

However, more specifications must still be met before the robot laser can safely make its precise incisions. For instance, the tiny, robot-guided endoscope where the laser is situated must remain immobile throughout the entire intervention, steadying itself inside the body. “That may sound trivial but in fact it’s a major technical challenge,” Cattin says.

A PhD student in the group led by co-project leader Georg Rauter has now found a solution for the problem: taking the knee joint as an example, she created a prototype for a fastening system that works using a type of balloon or air cushion. When these devices are put in place, they inflate and fill the space between the laser robot and the tissue, affixing the robot to the spot. “After the laser has made an incision, surgeons can let the air out of the cushions and the robot will move to the next incision site,” Cattin explains.

Step by step and piece by piece, the MIRACLE team are revolutionising bone surgery.