Innovating with Immersive Reality
Projects in this theme aim to develop novel applications of virtual reality and augmented reality to serve the unmet needs in industry verticals such as healthcare, education and entertainment.
In this project, we designed a cabin crew training system that utilizes text-to-speech and speech-to-text technology for service simulation, the Large Language Model (LLM) to simulate passenger responses, and Virtual Reality (VR) to recreate the cabin environment for better scalability and flexibility.
This project aims to enhance dentistry students' dentist-patient communication ability by developing a readily accessible software that provides risk-free environment for interactions with diverse geriatric patient archetypes, addressing the limitations of current standardised patient programmes.
The project aims to bridge the communication and workflow coordination gap among nurses while they are attending to emergency situations such as resuscitating a patient, through a VR simulation that serves as a platform for nursing students to practice these skills.
In this project, we designed an immersive AR portal to simplify the complexities of skincare product ingredients, enabling consumers to grasp their effects on the skin more effectively and visualise their impact on facial appearance.
This project aims to strengthen Pantene's brand engagement with the younger generation by introducing an innovative social media AR game/filter that not only addresses the challenge of connecting with a younger audience but also enhances their overall brand experience.
This project aims to value-add to the medical school's current episiotomy training procedure through a VR application that allows medical students to absorb episiotomy-related knowledge and skills in a realistic simulated environment.
Borderless is an immersive experience that enables people from different physical spaces to seemingly exist on the same plane, allowing strangers to self-reflect and forge social connections to boost well-being.
This project creates a virtual reality experience that offers an intimate walkthrough of Ng Eng Teng's former residence with thematically curated art pieces in his basement. The art pieces are showcased with interactive metadata and precisely recreated in virtual reality using 3D scanning technology in order to allow close-up interaction.
Mind Peek is AI-powered exhibit that can identify emotions in real time and provides a tranquil environment for visitors to unwind and reflect on their well-being and emotions.
This project aims to study if touch feedback can improve the performance of user interaction and user experience in VR world by conducting experiments such as button pressing, tracing contour and object selection using invisible ray-cast with a haptic glove developed by NUS.
The project aims to design a VR system to help interview candidates overcome interview anxiety by providing realistic interview scenarios, evaluating interview performance and providing accurate feedback during their preparation.
In this project, we designed an immersive medical education tool that offers hands-on training using virtual instructor hand guides, in-VR assessments, and data analytics, to bridge the gap between medical and midwifery students' theory and practical understanding so as to enhance their competency, due to declining birth rates, restricted access to delivery wards, and a shortage of instructors.
In this project, we designed a virtual reality simulation to provide medical students with opportunities to practise childbirth delivery procedures independently in a realistic setting.
This project aims to help defence lawyers reconstruct crime scenes through the use of VR technology to enable them and their clients to be able to immerse themselves in the scene from different perspectives at a more sustainable cost.
In this project, we created a virtual reality app to solve the problem of lack of practice partners for beginners who are learning a foreign language.
This project aims to design a tracking method for conventional ultrasound probes that enables 3D ultrasound reconstruction and mixed reality overlay on a patient’s anatomy.
Our project aims to design assistive technology for individuals with muscular deficiency that leverages on their gross-motor functions and minimises effort needed to perform communicative actions.
This project develops a novel AI conversational agent and hands tracking to allow orthopedic residents to use natural language to interact with the virtual patient and bare hands to execute physical examination. A dialog model was built from scratch using Rasa X, RASA Core and Microsoft Azure, and trained to handle the unpredictability of patient-doctor conversations.
Basic Cardiac Life Support training is essential for all medical staff and actively promoted to the general public. However, the training is manpower intensive and has limited availability to the public. We have developed a novel VR training solution using VR head mounted display, hand gloves and Resusci Anne Manikin, that could be used out-of-the-box without medical personnel intervention. User training data are logged and feedback is provided in real time. This project is developed in collaboration with Sengkang General Hospital and is funded by iHIS Research Innovation Enterprise Partnership grant.
This project aims to develop a birth delivery simulator targeted towards undergraduate medical students whom often do not have the opportunity to be in the delivery suite to perform hands-on procedures. They have to make do with a manikin for training, which is not realistic and does not train well for complications. This project intends to provide a realistic training toolkit through virtual reality with haptics for birthing simulations, from normal delivery to complication. This project is developed in collaboration with Department of Obstetrics & Gynaecology, NUH and NUS Yong Loo Lin School of Medicine.
Minimally invasive surgery (MIS) is becoming more popular due to its various benefits compared to conventional surgery. One of the ways to carry out MIS is using the da Vinci surgical system. However, this system has a steep learning curve and is not accessible. We have developed a virtual simulator incorporating VR system and the Geomagic Touch haptic device. We focused the training on soft tissue cutting which is an essential skill for robotic surgeons. This project is developed in collaboration with NUH Gynaecologic Robot Assisted Cancer & Endoscopic Surgery programme and NUS School of Computing.
Resident doctors at orthopedic surgery are seeking more training opportunity outside of operating room. With high-fidelity interactive stereoscopic recordings of spinal surgical procedures, we are able to transport the resident into the shoes of the operating surgeon. We do this by utilizing state of the art stereo cameras, and the Oculus Quest, with its built in positional tracking systems. Resident doctor can also review the lecture notes and 3D spine model in VR. This project is developed in collaboration with the Department of Orthopaedic Surgery at NUH and NPLASTY.
Museum Alive project explores innovative technology to transform museum experience, provide additional contexts to the paintings and promote dialogue and interaction among visitors. What experiences did the painter go through in his or her life? What was the political situation at the time they were painting it? These, and many other contexts, add depth to a painting. An Augmented Reality app was developed to track paintings in real time. Additional contexts are overlaid on demand. New context hotspots could also be added to the cloud database through the app, promoting an open dialogue among visitors.
Learning martial art movement requires thorough practice and correction, especially for novice. In this Wushu project, we hypothesised that movement can be learned more effectively if feedback on the movement can be provided to the learner in a direct way using a virtual ghost model of the instructor. The instructor’s moves are pre-recorded using wearable VR trackers. The movements are played in segments with selectable playback speed in a form of ghost model to the trainees. The VR system also provide real time feedback on the accuracy of the body movement and total score at the end of each exercise.
Fire is a hazard that can happen anywhere and everyone should be prepared for it. However, hands-on training is costly in terms of manpower and resources and is limited due to the potential hazards of an actual fire. For the general population, infrequent hands-on practices and non-hands-on training materials limit their preparedness.The advantages of the Virtual Reality based training tool include reducing manpower and resource needs, not having to set up a real fire and increasing the accessibility to a minimum standard of hands-on practice.
This project explores the usage of virtual reality in aircraft maintenance. Integration of virtual reality in aircraft maintenance training helps to reduce the training costs while ensuring a safe environment for trainees. The developed prototype combines visuals, haptics, movement and spatial sound cues in a head gear to create a realistic virtual environment for training.