International team develops advanced COVID-19 virtual reality training simulations

By Melissa Shaw

Advanced virtual training simulations are being created by an international tech team based in Vancouver to train healthcare workers on the correct application of personal protective equipment (PPE) in long-term care facilities. 

Post-doctorate researchers and graduates from Simon Fraser University's School of Interactive Arts and Technology (SIAT) and Virtro, a Vancouver-based tech company, are collaborating to develop virtual reality (VR) and artificial intelligence (AI) based training simulations with Australia's Monash University and the Victorian Department of Health and Human Services.

"COVID-19 has made proper face-to-face training incredibly difficult for programs for professionals, including healthcare workers and medical staff, which require lab training and practicums," says SIAT Professor Steve DiPaola, who specializes in VR and AI and oversees SIAT’s iVizLab. 

"SIAT's AI and VR research efforts along with Virtro's extensive work in developing AI characters, called Virtual Humans, can help break the existing training bottleneck, and give both practicing healthcare workers and students a safe and real-world experience in VR."

The international team is developing an application that leverages Virtro’s proprietary conversation engine, allowing learners to have natural and unscripted conversations with Virtual Humans. Learners will practice communicating with Virtual Humans on the proper PPE application procedures.

"We're able to replicate what a potential healthcare worker or nurse is going to face when they go out and deal with patients," says Virtro President Lee Brighton. "Virtro delivers purpose-based learning using AI and VR technology that delivers faster learning with higher engagement and retention. Students using our technology feel a direct emotional connection to the experience, and the simulation becomes far more lifelike and memorable."

Virtro has won several grants to partner with DiPaola and his team. His groundbreaking research includes AI-based computational models such as his Computer Modelling Human Expression and Emotion Gesture Tracking. 

In addition to the PPE training simulation, Virtro and iVizLab are currently developing a method that can analyze movement complexities when medical staff interact with patients on a physical level. 

Virtro has hired more than a dozen SFU graduates with many from DiPaola's VR classes. Ioana Sandor, a recent SIAT graduate and developer at Virtro, says, "I'm very fortunate to be part of a team conducting leading-edge research and development with technology to address an issue that COVID-19 has created globally.”

A prototype of the PPE training simulation is expected to be available early next week. Talks are underway to expand the training to workers in a variety of healthcare fields in Canada.