Surgery simulators are key to assessment of trainees

AI and virtual reality can determine neurosurgeon expertise with 90% accuracy

Machine learning-guided virtual reality simulators can help neurosurgeons develop the skills they need before they step in the operating room, according to a new study.

Research from the Neurosurgical Simulation and Artificial Intelligence Learning Centre at the Montreal Neurological Institute and Hospital (The Neuro) and McGill University shows that machine learning algorithms can accurately assess the capabilities of neurosurgeons during virtual surgery, demonstrating that virtual reality simulators using artificial intelligence can be powerful tools in surgeon training.

Fifty participants were recruited from four stages of neurosurgical training; neurosurgeons, fellows and senior residents, junior residents, and medical students. They performed 250 complex tumour resections using NeuroVR, a virtual reality surgical simulator developed by the National Research Council of Canada and distributed by CAE, which recorded all instrument movements in 20 millisecond intervals.

Using this raw data, a machine learning algorithm developed performance measures such as instrument position and force applied, as well as outcomes such as amount of tumour removed and blood loss, which could predict the level of expertise of each participant with 90 per-cent accuracy. The top performing algorithm could classify participants using just six performance measures.

Machine learning-guided virtual reality simulators can help neurosurgeons develop the skills they need before they step in the operating room. Photo credit: Helmut Bernhard/The Neuro

This research, published in the Journal of the American Medical Association on Aug. 2, 2019, shows that the fusion of AI and VR neurosurgical simulators can accurately and efficiently assess the performance of surgeon trainees. This means that AI-assisted mentoring systems can be developed that focus on improving patient safety by guiding trainees through complex surgical procedures. These systems can determine areas that need improvement and how the trainee can develop these important skills before surgeons operate on real patients.

“Physician educators are facing increased time pressure to balance their commitment to both patients and learners,” says Dr. Rolando Del Maestro, the lead author of the study. “Our study proves that we can design systems that deliver on-demand surgical assessments at the convenience of the learner and with less input from instructors. It may also lead to better patient safety by reducing the chance for human error both while assessing surgeons and in the operating room.”

Research funding for this study was provided by Franco Di Giovanni Foundation, the Montreal English School Board, the McGill Department of Orthopedics, the Fonds de recherche du Québec – Santé (FRQS), and a Robert Maudsley Fellowship for Studies in Medical Education from the Royal College of Physicians and Surgeons of Canada.