Synthetic data generation pipeline for geometric deep learning in architecture

S. Fedorova, A. Tono, M. S. Nigam, J. Zhang, A. Ahmadnia, C. Bolognesi, Dominik L. Michels

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


With the growing interest in deep learning algorithms and computational design in the architectural field, the need for large, accessible and diverse architectural datasets increases. Due to the complexity of such 3D datasets, the most widespread techniques of 3D scanning and manual building modeling are very time-consuming, which does not allow to have a sufficiently large open-source dataset. We decided to tackle this problem by constructing a field-specific synthetic data generation pipeline that generates an arbitrary amount of 3D data along with the associated 2D and 3D annotations. The variety of annotations, the flexibility to customize the generated building and dataset parameters make this framework suitable for multiple deep learning tasks, including geometric deep learning that requires direct 3D supervision. Creating our building data generation pipeline we leveraged the experts' architectural knowledge in order to construct a framework that would be modular, extendable and would provide a sufficient amount of class-balanced data samples. Moreover, we purposefully involve the researcher in the dataset customization allowing the introduction of additional building components, material textures, building classes, number and type of annotations as well as the number of views per 3D model sample. In this way, the framework would satisfy different research requirements and would be adaptable to a large variety of tasks. All code and data is made publicly available:
Original languageEnglish (US)
Title of host publicationThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
PublisherCopernicus GmbH
Number of pages8
StatePublished - Jun 28 2021

Bibliographical note

KAUST Repository Item: Exported on 2021-10-14
Acknowledgements: We would like to thank Andrea Giordano, Georgia Gkioxari, Kaichun Mo, Silvio Savarese, Martin Fischer, Andrea Tagliasacchi, Nicolas Chaulet, Lamberto Ballan, Dmitry Kudinov, Mohammed Keshavari, Andrean Zani, Ignacio Garcia Dorado for valuable discussions and inputs. Furthermore, we are grateful and thankful to the Pytorch, Pytorch3d communities for their help and support. Finally, we would like to acknowledge contributions through helpful discussions, technical support, GPUs donations and Hardware donated by NVIDIA, and the Computational Design Institute.


Dive into the research topics of 'Synthetic data generation pipeline for geometric deep learning in architecture'. Together they form a unique fingerprint.

Cite this