Utilizing Mixed Reality Glasses as a Computer Peripheral Device for Users with Mobility Disabilities
As part of an individual project for a graduate-level course on Mixed Reality Applications in Medicine, I developed an App that uses a Mixed Reality Head Mounted Display (HMD) (HoloLens 2) to accomplish mouse and keyboard actions on a PC for individuals with motor disabilities. To accomplish this, I interfaced the firmware of both the PC and HMD to utilize Gaze and Voice input from the HMD to command cursor positioning and clicking on the PC via wireless communication. The goal of the project was to address common occupational injuries and soreness from body overuse.
This work was presented at the 2021 Stanford Medical Mixed Reality Symposium. Here is a video of me presenting the work. The project was also featured in a Wu Tsai Neurosciences Institute article about medical mixed reality applications.
Here is a brief clip of me testing the app during development
Demo