Vision-language models (VLMs) have made significant advances in understanding images and videos, but accurately reasoning about spatial relationships remains challenging. In particular, models may rely heavily on visual appearance while underutilising important geometric information.
The paper introduces GeoSR, a framework designed to encourage VLMs to make better use of geometric information for spatial reasoning in both static images and dynamic videos. GeoSR incorporates two key techniques. Geometry-Unleashing Masking strategically masks portions of 2D visual information during training, encouraging the model to rely more on geometric cues. Geometry-Guided Fusion then uses a learned gating mechanism to adaptively strengthen geometric information where it is most relevant. Through these approaches, GeoSR enables VLMs to more effectively leverage 3D geometric cues when reasoning about spatial relationships. Experiments on static and dynamic spatial reasoning benchmarks demonstrated consistent improvements over existing geometry-aware approaches, establishing new state-of-the-art performance on the reported benchmarks.
The research highlights how incorporating geometric understanding can strengthen the ability of AI systems to reason about the physical world, with potential applications in areas such as embodied AI, robotics and intelligent systems.
