Congratulations to team ‘EgoVLP’ who won First Place Awards in both the Object State Change Classification on Ego4D Challenges 2022 and Multi-Instance Action Retrieval on EPIC-Kitchens Challenges 2022, held by IEEE/CVF Computer Vision and Pattern Recognition (CVPR). The team was also invited to give two talks at the CVPR Joint 1st Ego4D and 10th EPIC Workshop held on 19-20 June 2022.
The core members of ‘EgoVLP’ who won the awards are from Show Lab@NUS which focuses on egocentric vision-language understanding research and supervised by Assistant Professor Mike Zheng Shou from the Department of Electrical and Computer Engineering.
The IEEE/CVF CVPR is the top conference focused on the fields of computer vision and pattern recognition and has been held since 1983, which is regarded as one of the most influential conferences in artificial intelligence.
The Ego4D is a massive-scale Egocentric video dataset of unprecedented diversity. The Object State Change Classification challenge aims to understand human interactions with objects. Given a short video clip, this task aims to recognize whether a short video clip has a state change, e.g., the state change of turning on a machine or chopping a tomato into pieces, which is an essential aspect of creating a human-centric AI assistant system.
The EPIC-Kitchens is a large-scale dataset in egocentric vision that focuses on daily activities in the kitchens. The Multi-Instance Retrieval challenge is a task of video-text retrieval, focused on exploring the video and language interactions in real-world scenarios. E.g., given a text query like ‘‘cutting a pizza using a cutter’’, aims to retrieve the video clip containing relevant actions like ‘‘slicing a pizza’’. Conversely, a video clip can also be used as a query to find relevant captions. This challenge contributes to multimodal AI understanding of visuals and linguistics.
