Korey Thomson, a third-year student in the Instrumentation and Control Engineering Technology program is creating software to run virtual holograms for clinical simulations for medical students at the Queen’s University Medical Simulation Lab. The software will be used on a headset called the HoloLens which uses 3D models to create learning scenarios for medical students. Learn more about this collaborative project in an article written by Korey.
Story By Korey Thomson
The HoloLens Patient Project, created with Queen’s University Clinical Simulation Centre technicians and doctors, uses the Microsoft HoloLens 2 headset to create a virtual hologram of a patient for medical training. This patient can show various medical symptoms for participants to diagnose and suggest treatments. Current symptoms are sweating, complexion, abnormal chest motion and breathing, and myoclonic and tonic seizures. The patient can also detect and look at the participant to add immersion.
The project is being developed for the Clinical Simulation Centre at Queen’s University. The goal is to create a cost-effective and scalable system to run medical training scenarios. The use of a headset and a virtual patient removes the reliance on expensive medical manikins which tend to be static in functionality and do not scale well. The HoloLens allows these scenarios to be created to the exact specifications of the instructor without the need for physical equipment.
The scenarios consist of a 3D model to represent the patient. This model has pre-defined animations which can be triggered by a technician. For example, if the technician turns on a seizure the model starts to play through the seizure animation. If no animations are on, the patient defaults to laying down. The technician can also control the pallor of the patient through a sliding scale, as the value increases the model starts to look increasingly pale. This can also be done with diaphoresis, or sweat, the higher the value the more light reflects off the model to simulate sweat.
A system like this will allow future medical students to practice diagnosing patients virtually in a safe environment. The relatively low cost of a HoloLens headset at roughly $5,000 makes this an attractive option for simulation facilities where a manikin can cost upwards of six figures. The system is also highly scalable and can integrate other systems like artificial intelligence to improve functionality and create truly autonomous and unique patients that respond differently to stimulation creating a more dynamic training environment.
As technology improves, augmented reality training scenarios will only become more immersive. The current drawback for medical training with the HoloLens is the lack of physical feedback as participants cannot touch the hologram. This removes checking for a pulse or chest compressions from the capability of the HoloLens system but, as technology advances, this will surely become possible.
"We value the opportunity to partner with SLC on their co-op program. Korey’s work is directly contributing towards our project team’s ongoing development of augmented reality as a medium which is revolutionizing clinical simulation training," said Mitch Doherty, Operations Manager, Clinical Simulation Centre, School of Medicine, Queen’s University.