Training 3D-AI Models for the Early-Stage Architectural Design Phase

Instructor: Adam Sebestyen

Institute for Architecture and Media, T.U. Graz, Austria

Email: sebestyen@tugraz.at

Duration: 2 days

Mode of participation: Physical

Brief Description

This workshop aims at teaching participants how to create a custom 3D dataset large enough to train their own generative deep neural network. Using state-of-the-art 3D parametric software tools like Grasshopper, participants will generate the necessary datasets. These datasets will then be used to train a generative deep learning neural network based on Variational Autoencoders (VAEs), developed at the Institute for Architecture and Media and the Graz Center for Machine Learning at Graz University of Technology, Austria.

Specifically, this year’s workshop builds upon the foundation laid during the eCAADe 2023 workshop, “Build and Train Your Own 3D AI-Generative Design Tool.” In the previous iteration, participants trained diffusion models with their own datasets. Since then, we have made advancements in our tools and have updated the workshop accordingly. Instead of using the slow-to-train diffusion model, we will employ fast trainable Variational Autoencoder (VAE) models. This modification enables us to train more models during the workshop, providing participants with increased exposure to the underlying technology and allowing them ample time to experiment with various trained AI models. Additionally, we have introduced new methods for representing 3D models to AI through a signed distance function, replacing the proprietary volume-based method used last year. Furthermore, we incorporate methods to let the workshop participants control the AI generation process thus finding novel and unexpected design solutions.

By the end of the workshop, participants will have created and trained their own custom AI-based design tools capable of synthesizing new 3D outputs, specifically tailored for early-stage architectural design.

Program

The workshop will be divided into two main parts, each spanning one of the two workshop days:

Part 1: Generation of Custom Datasets (Grasshopper)

Participants will utilize parametric design tools to create custom datasets suitable for training their own generative deep learning network. The workshop will commence with an overview of key principles regarding neural network training and dataset creation, emphasizing the importance of usable 3D data representations. Each participant will craft their unique dataset. These individual datasets will then be combined to train neural networks. The resulting trained network, based on the PyTorch deep learning library, will possess the capability to generate hybrids and complex combinations of the individual datasets.

Part 2: Training the Neural Network and Generating New Outputs (Python)

The combined datasets will serve as the basis for training our VAE AI model. Following the training process, participants will learn how to generate new outputs and convert them back into 3D mesh data. Throughout this phase, participants will be encouraged to explore various dataset combinations and examine the resulting generative outputs. Finally, participants will present their generated hybrids to the group.

Required Skills, Hardware, Software

We will use Grasshopper to generate the datasets. Some basic knowledge of parametric design is recommended. As we will see, even very simple parametric scripts can be used to train neural networks that can produce complex outputs.

Will provide the participants with the VAE AI model which is a custom Python script. Google Colab will be used to train and execute the AI. The free version of Google Colab can be used for this workshop. However, we would recommend buying a few compute-units (100 compute-units for €11 should be more than enough) to do training of more complex models. Further a Google Drive account with sufficient space for data and AI models is required (about 5 to 10 GB). Python knowledge is not required to run the script. Since Google Colab runs in the cloud, participants only need a laptop that can run Grasshopper.