This studio is dedicated to exploring challenges and opportunities at the intersection of immersive technologies, generative AI and large language models (LLMs). The studio investigates current limitations of VR/AR simulation systems used in medical and aviation training, designs new ways to make them more immersive and effective and explore new use cases. User experience and usability will also be a key focus. Students in this studio have the opportunity to work on the design, development, and/or user research of a chosen topic or problem statement. Possible areas of exploration include the following:

• VR/AR applications for medical and aviation training
• Simulating realistic AI characters using generative AI/LLM
• Generative AI/LLM for auto feedback and assessment in medical training without need for human evaluators
• Computer vision and machine learning for object detection and 3D reconstruction
• Simulating emotion in virtual human and sensing user emotional responses

Students may choose one of the following problem statements or self-propose a new topic in this studio:

• VR/MR childbirth delivery simulator (in collaboration with O&G department)
• Virtual paediatric (infant) patient simulation (in collaboration with Centre for Healthcare Simulation)
• AI geriatric (elderly) patient for dental training (in collaboration with Faculty of Dentistry)
• AI social companion robot/pet for elderly (in collaboration with NUS Nursing)
• Interprofessional training for nursing and medical students (in collaboration with NUS Nursing)
• AI co-pilot in VR simulator for pilot competency training (in collaboration with Singapore Airlines-NUS Digital Aviation Corp Lab)
• Realistic AI passenger simulation for cabin crew training (in collaboration with Singapore Airlines-NUS Digital Aviation Corp Lab)
• Emerging applications of GenAI and VR/MR

Project partners:

• National University Health System
• Yong Loo Lin School of Medicine
• Faculty of Dentistry
• Singapore Airlines-NUS Digital Aviation Corp Lab

Project supervisor:

• A/Prof Khoo Eng Tat (etkhoo@nus.edu.sg)

Studio timeslot in Semester 2 AY2024/2025:

• Wednesday 9 am to 12 pm @ iDP Studio 5 (E2A-04-02)