Multi-Sensory Immersive Reality for Cultural Heritage Computing
The integration of innovative technology in cultural heritage sites, such as virtual audio-visual reconstructions, has become a prevalent approach to enhance the communication and perception between these spaces and their visitors. Modern immersive reality technologies, including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), have significantly transformed the entertainment and educational experiences at cultural heritage sites. These technologies offer substantial benefits for exhibition enhancement, historical reconstructions, and interactive exploration. Since people naturally use all five senses — visual, auditory, olfactory, tactile, and gustatory to experience their surroundings, therefore, beyond the typical audio-visual content provided by immersive technologies, our aim is to incorporate haptics, scents, and gustatory elements to create truly multi-sensory immersive environments. We aim to deliver a more engaging, immersive and embodied cultural heritage experience, addressing the limitations of traditional cultural heritage perception and preservation methods, which often rely on static displays and minimal interactivity.
The studio's scope includes developing a platform that uses immersive reality (VR/AR/MR) and advanced hardware (e.g. wearable and tangible devices) to explore and create immersive cultural heritage experiences. Students can work on the following problem statements or propose new ones:
- Tangible cultural heritage: How can immersive reality and advanced hardware be used to reconstruct heritage sites and digital artefacts, allowing users to interact with them using physical sensations (e.g., haptics)?
- Intangible cultural heritage: How to create immersive, embodied experiences that transport users into historical moments (visual), stories (audio), foods (smell and taste), handcrafts (haptics), and more to explore cultural lifestyles?
- Natural heritage: How can we digitally recreate Singapore's countryside and natural environments, such as its maritime heritage and the Kranji countryside, to showcase the unique blend of natural and cultural landscapes?
- Accessibility: How can the platform be designed to be accessible for the public, researchers, and educators, making it a valuable resource for schools, museums, and cultural institutions to deepen understanding of our shared history?
- Long-term preservation: What strategies can be implemented for the long-term preservation and maintenance of tangible, intangible, and natural cultural heritage data to ensure its availability for future genera7ons?
This studio aims to bridge the gap between our rich cultural heritage and emerging technologies, making our history, landscape and culture more engaging, accessible, embodied and relevant to the people of Singapore community and the global community.
Project partners: NUS Libraries, NUS Faculty of Arts and Social Sciences
Project supervisor:
- Dr Cai Shaoyu (shao.cai@nus.edu.sg)
Studio timeslot in Semester 2 AY2024/2025:
- Thursday 1 pm to 4 pm @ iDP Studio 5 (E2A-04-02)