Generative AI in mixed reality for cabin crew training

The current training method for cabin crew in commercial airlines faces inefficiency and scalability challenges due to fixed mock-up capacities and diverse plane models requirements, prompting the need for innovative solutions.

Virtual reality (VR) emerges as a reliable alternative, offering mobility, flexibility, and cost-effectiveness. We leverage the power of a Large Language Model (LLM) to analyse and generate responses as the virtual passenger. This allows us to generate a more unpredictable situation yet able to provide objective feedback to the trainees through tracking of various data. Speech-to-Text (STT) and Text-to-Speech (TTS) algorithms are also utilised to analyse and generate speech that is more closely aligned with a Singaporean setting. This project explores the integration of these technologies to create a convincing training environment with accurate and natural interactions, addressing the pressing demand for scalable and effective cabin crew training solutions.

youtube-video-thumbnail

Project Team

Students:

  • Chen Qing Hua (Mechanical Engineering, Class of 2024)
  • Chen Yuhan (Electrical Engineering, Class of 2024)
  • Raymond Bala (Electrical Engineering, Class of 2024)

Supervisor: