A new study from Unity Health Toronto researchers evaluates the use of CHARTWatch, the artificial intelligence (AI) early warning system developed at St. Michael’s Hospital that monitors hospitalized patients in real-time and identifies those at high risk of unexpected death or transfer to an intensive care unit (ICU) and sends alerts so that doctors and nurses can intervene early.

The study, published in the Canadian Medical Association Journal on Sept. 16, shows a significant 26 per cent reduction in unanticipated mortality after the tool was implemented on the St. Michael’s general internal medicine (GIM) ward.

“As AI tools are increasingly being used in medicine, it is important that they are evaluated carefully to ensure that they are safe and effective,” says lead author Dr. Amol Verma, General Internist at St. Michael’s who led the development and implementation of CHARTWatch. 

“Our findings suggest that AI-based early warning systems are promising for reducing unexpected deaths in hospitals.”

One of the primary sources of unplanned admission to the ICU is the unexpected deterioration of hospitalized patients, which prompted researchers at Unity Health Toronto and the University of Toronto to develop this AI tool and study its effectiveness. 

“This important study evaluates the outcomes associated with the complex deployment of the entire AI solution, which is critical to understanding the real-world impacts of this promising technology,” says co-author Dr. Muhammad Mamdani, Vice President of Data Science and Advanced Analytics at Unity Health Toronto. 

“We hope other institutions can learn from and improve upon Unity Health Toronto’s experiences to benefit the patients they serve. Unity Health Toronto is a collaborative leader already helping to spread our AI tools via innovative partnerships with more to come.”

The study analyzed data from 13,649 patients aged 55 to 80 years old admitted to the GIM unit (9,626 in the pre-intervention period and 4,023 using CHARTWatch) and 8,470 admitted to subspeciality units that did not use CHARTWatch.

Helping to prioritize patient needs

“The CHARTWatch project started at Unity Health when we asked patients, clinicians, hospital leaders, what would you want to use artificial intelligence for? If you could predict one thing that AI would tell you, what should that be?” says Verma. 

“And one of the leading priorities of everyone was to be able to predict in advance which patients might become so sick in hospital that they need ICU or might die.” 

During the 19-month-long intervention period, 482 patients in the GIM became high-risk, compared with 1,656 patients who became high risk in the 43-month-long pre-intervention period. There were also fewer non-palliative deaths in the CHARTWatch group than in the pre-intervention group (1.6 per cent versus 2.1 per cent). 

“If I were a patient, I would be so relieved to know that we have this kind of system,” said co-author Dr. Yuna Lee, Division Head and General
Internist at St. Michael’s.

“So when the patient gets a high risk alert, they’re going to be assessed by senior staff right away. And also they’re also going to have quite close monitoring by nursing staff, so they check on them every one to two hours.”

Expanding the impact of CHARTWatch

CHARTWatch inputs more than 100 aspects of a patient’s medical history and current health status that are routinely stored in the hospital’s electronic medical record. It analyzes the interactions between these inputs and how they change over time. With that information, it’s able to categorize each patient by their risk for deterioration and send an alert to prioritize treatment. 

“The team lead receives a high risk CHARTWatch alert and then she would inform the primary nurse,” says Emme Rose Villanueva, Registered Nurse at St. Joseph’s Health Centre. “Then we would closely monitor the patient, prioritize high risk alert versus the stable ones, and do necessary
interventions as needed.” 

The study is one of the first rigorous studies evaluating how CHARTWatch can benefit hospital patients in Canada and shows the potential real world impact of AI on the health care sector.

“Every time I hear the alarm I tell the nurses, whoever has the patient, and then also inform the doctor,” said Alicia Feruelo, Team Leader and Registered Nurse at St. Joseph’s Health Centre. “From the first time it’s been used, I could only remember one transfer to ICU.”

Damian Jankowicz, Vice President and Chief Information & AI Officer, says he hopes AI tools like CHARTWatch will continue to have a profound impact on patients. 

“Hopefully with reduced administrative burden on our providers, they will have more time to spend with our patients and really focus on the patient needs,” he says. 

“I hope that AI will be able to distill the incredible amounts of information that’s coming at our clinicians into important components and really bring their clinical judgment to the
forefront.”