Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario
Article Sidebar
Main Article Content
Abstract
Background: Stethoscopes and mannequins are basic tools for medical practitioners to diagnose
various problems. Restricted by space, resources, and time, these tools and common video training
are sometimes inaccessible. These obstacles may be solved by immersive virtual reality (VR) that
improves interaction and blurs the line between virtual and reality. Therefore, this study combines
virtual and real interaction technologies to investigate the impact of multi-sensory interaction on
learning outcomes in chest auscultation.
Methods: This study integrated VIVE Tracker technology to provide synchronized tactile stimulation
in VR. University students in medical-related disciplines participated in the study. Interviews
were conducted to understand how students perceived the effectiveness and usefulness of the
thoracic auscultation VR.
Conclusion: Most students expressed that tactile stimulation gave realistic sensations and
effectively reinforced the immersive effect. Simultaneously, it increased their interest and
motivation in learning. The integration of synchronized tactile stimulation is a novel approach
that expands the realm of interactivity beyond handheld controller manipulation and creates a
platform to simulate medical-related scenarios to learn complex knowledge
and skills.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
References
Hsieh MC, Lin YH. VR and AR applications in medical practice and education. The Journal of Nursing. 2017;64(6):12-18.https://10.6224/JN.000078
Matthews S, Wood K, Quevedo AJU, Jaimes N, Dubrowski A, Kapralos B, Alam F, Rojas D. Work-in-Progress–A preliminary eye tracking and HMD orientation comparison to determine focus on a cardiac auscultation training environment. In: Proceedings of the 7th International Conference of the Immersive Learning Research Network (ILRN); 2021; Eureka, CA, USA.https://10.23919/iLRN52045.2021.9459383
Yeh HF. Virtual reality skills training trends in nurse practitioner education. The Journal of Nursing. 2021;68(5):13-17. https://10.6224/JN.202110_68(5)
Chang YM, Lai CL. Exploring the experiences of nursing students in using immersive virtual reality to learn nursing skills. Nurse Education Today. 2021;97. https://10.1016/j.nedt.2020.104670
Hara CYN, Goes FDSN, Camargo CRAA, Fonseca LMM, Aredes NDA. Design and evaluation of a 3D serious game for communication learning in nursing education. Nurse Education Today. 2021;100. https://10.1016/j.nedt.2021.104846
Bagher MM, Bagher P, Bagher J, Bagher P, Klippel A. Move the object or move the user: The role of interaction techniques on embodied learning in VR. Frontiers in Virtual Reality. 2021;2. https://10.3389/frvir.2021.695312
Hannans JA, Nevins CM, Jordan K. See it, hear it, feel it: Embodying a patient experience through immersive virtual reality. Information and Learning Science. 2021;122(7-8):565-583. https://10.1108/ILS-10-2020-0233
Moon J, Jeong M, Oh S, Laine T, Seo J. Data collection framework for context-aware virtual reality application development in Unity: Case of avatar embodiment. Sensors. 2022;22(12).https://10.3390/s22124623
Pratviel Y, Bouni A, Veronique DA, Larrue F, Arsac LM. Avatar embodiment in VR: Are there individual susceptibilities to visuo-tactile or cardio-visual stimulations? Frontiers in Virtual Reality. 2022;3.https://10.3389/frvir.2022.954808
Salagean A, Crellin E, Parsons M, Cosker D, Fraser DS. Meeting your virtual twin: Effects of photorealism and personalization on embodiment, self-identification and perception of self-avatars in virtual reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ‘23); 2023. p. 1-16. https://10.1145/3544548.3581182
Škola F, Tinková S, Liarokapis F. Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Frontiers in Human Neuroscience. 2019;13. https://10.3389/fnhum.2019.00329
Ao D, Wu W, Guo X. Immersion and intersectionality - Virtual reality in cross cultural art exhibition courses. In: Rau PLP, editor. Cross-Cultural Design. HCII 2023. Lecture Notes in Computer Science, vol. 14024. Springer; 2023. https://10.1007/978-3-031-35946-0_16
Bowman DA, McMahan RP. Virtual reality: How much immersion is enough? Computer. 2007;40(7):36-43. https://10.1109/MC.2007.257
Moffitt RL. A psychosocial investigation of exercise preferences in real and virtual environments. Psychology of Sport and Exercise. 2023; 70:102530. https://10.1016/j.psychsport.2023.102530
Gammage KL, Lamarche L, Drouin B. Self-presentational efficacy: Does it moderate the relationship between social physique anxiety and physical activity in university students? International Journal of Sport and Exercise Psychology. 2014;12(4):357-367. https://10.1080/1612197X.2014.932824
Dirin A, Laine T. The influence of virtual character design on emotional engagement in immersive virtual reality: The case of feelings of being. Electronics. 2023;12(10). https://10.3390/electronics12102321
Gani A, Pickering O, Ellis C, Sabri O, Pucher P. Impact of haptic feedback on surgical training outcomes: A randomised controlled trial of haptic versus non-haptic immersive virtual reality training. Annals of Medicine and Surgery. 2022;83. https://10.1016/j.amsu.2022.104734
Kanakamedala CA, Dankert FJ, Parola R, Egol AK, Aggarwal KV, Lajam MC. Haptic feedback during virtual reality training significantly improves first-year orthopedic resident performance at tibia drilling: A randomized trial. Current Orthopaedic Practice.2023;34(5):251-256. https://10.1097/BCO.0000000000001223