Follow CDE
PDF Download
A cross-platform system lets designers generate AI-driven 3D forms on a tablet, then step inside them at full scale — turning how early-stage architecture is conceived, evaluated and felt on its head.
Artificial intelligence (AI) can now conjure a cathedral a few blinks of an eye. Type a few words into Midjourney or DALL·E and the screen fills with soaring Gothic ribbed vaults, light-drenched atriums and sweeping cantilevers. The visuals are breathtaking — though completely uninhabitable. You cannot walk through them, sense their scale or feel the weight of a ceiling pressing overhead. It is the crux of many architects’ frustration when it comes to generative AI: it produces visual spectacle but offers no spatial “awareness.”
Assistant Professor Gabriel Lipkowitz and his team developed a multimodal system that lets designers generate AI-driven 3D forms on a tablet.
Assistant Professor Gabriel Lipkowitz from the Division of Industrial Design at the College of Design and Engineering, National University of Singapore, wants to close that gap. As the Principal Investigator of the Interactive 3D Design Lab, he has developed a multimodal system that connects two devices — Apple’s iPad for rapid AI-driven 3D generation and the tech giant’s spatial computing headset, Apple Vision Pro, for fullscale spatial immersion — into a single, fluid design workflow which enables designers to step inside their AI-generated spaces.
Across three user studies involving 50 architecture students and expert practitioners, the research found that embodied interaction with AI-generated forms — walking through, around and within them — changes how designers judge proportion, perceive enclosure and make decisions about mass and light. These are qualities that flat screens simply cannot convey.
Flat screens, one-dimensional thinking
The problem the team solved precedes generative AI, but AI has thrust it into the limelight. Architectural design has always involved a leap of imagination, from a sketch or a 3D model on screen to the perceived reality of a built space. Current AI image generators have made the first half of that leap almost effortless, producing detailed concepts at extraordinary speed.
"We wanted to reposition it from a detached visual tool to something that is embedded inside the designer’s experiential process.”
Yet the output is still just a picture. Even recent 3D generative systems, which add depth to AI-produced forms, struggle to convey accurate proportions to designers and clients.
“AI continues to function as an external generator of form rather than a collaborator in design thinking,” says Asst Prof Lipkowitz. “We wanted to reposition it from a detached visual tool to something that is embedded inside the designer’s experiential process.”
"We wanted to reposition it from a detached visual tool to something that is embedded inside the designer’s experiential process.”
"We wanted to reposition it from a detached visual tool to something that is embedded inside the designer’s experiential process.”
Sketch, generate, inhabit
The system built by his team works in two complementary modes. On the iPad, designers begin with the familiar: sketching, typing text prompts or uploading reference images. The app generates 3D models from these inputs, which can be rotated, scaled and refined with touch gestures. It is fast, portable and low-commitment — ideal for the messy, exploratory phase when ideas are
still blossoming.
The second mode is where the ground shifts. Through Apple Vision Pro, those same AI-generated models appear at full, one-to-one scale in the designer’s physical environment. A massing study (a “big picture” of a building’s 3D structure) that looked compact on a tablet screen suddenly towers overhead. A courtyard that seemed generous on the iPad reveals itself, once inhabited, as cramped. Walls close in. Light hits differently. The design has become experiential.
The two platforms also support collaboration: multiple users on iPads and Apple Vision Pros can join the same session, where they view and edit 3D models in real time. One designer generates, another rescales, a third critiques — a spatial desk crit, mediated by AI.
Step into my world
The team evaluated the system across three studies of increasing specificity. In the first, twelve participants explored both interfaces freely. Students described walking through AI-generated forms as “eye-opening,” revealing proportions and volumes invisible on screen. The iPad, meanwhile, drew praise for its speed and intuitive touch-based adjustments, and participants likened it to sketching on paper, but faster.
A second study with 30 participants compared the prototypes against conventional workflows: online image searching and standalone AI image generation. The iPad scored highest for accessibility and ease of use. The Apple Vision Pro earned the highest ratings for alignment with the designer’s mental image and overall satisfaction. Participants consistently reported that immersion enhanced their understanding of massing, enclosure and proportion — spatial qualities that screen-based tools left abstract.
The third study, with eight participants, pitted the system directly against traditional design methods such as hand sketching and the industry-standard Rhino modelling. Designers using the prototype produced more variations in the same timeframe and reached viable early-stage concepts faster. Rhino retained its edge for precision, but its complexity and setup time constrained the freewheeling ideation the prototype encouraged. Participants described the system as a complementary extension of existing tools, particularly effective for massing and early communication with clients and peers.
Expert reviewers, including an architecture professor and a practising architect, reinforced this view. From an educational standpoint, they noted the system’s potential to help students overcome the daunting blank page, externalise rough ideas quickly and critically evaluate AI outputs rather than passively accepting them. “If students can quickly visualise a rough idea before investing hours modelling in Rhino, it could change how early studio works,” one reviewer observed.
The prototypes are not without limitations. AI-generated models tend towards the volumetric and schematic, lacking architectural detail such as openings or articulated surfaces. In addition, the participant pool skewed towards students; broader professional validation remains ahead.
"While we don’t intend to replace the architect’s hand or eye, we want to extend the moment before form is fixed, keeping ideas fluid, spatial and open to critique for longer.”
"While we don’t intend to replace the architect’s hand or eye, we want to extend the moment before form is fixed, keeping ideas fluid, spatial and open to critique for longer.”
"While we don’t intend to replace the architect’s hand or eye, we want to extend the moment before form is fixed, keeping ideas fluid, spatial and open to critique for longer.”
Asst Prof Lipkowitz and his team are now working to address these gaps, developing architecture-specific AI models capable of finer geometric detail and fully editable 3D outputs that integrate with existing workflows. "While we don’t intend to replace the architect’s hand or eye, we want to extend the moment before form is fixed, keeping ideas fluid, spatial and open to critique for longer,” adds Asst Prof Lipkowitz. “In a discipline where body and space intertwine, that may be exactly where AI belongs.”
Read More
View Our Publications ▏Back to Forging New Frontiers - May 2026 Issue
If you are interested to connect with us, email us at cdenews@nus.edu.sg
