Creating consistent scene graphs using a probabilistic grammar

Tianqiang Liu*, Siddhartha Chaudhuri, Vladimir G. Kim, Qixing Huang, Niloy J. Mitra, Thomas Funkhouser

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

58 Scopus citations


Growing numbers of 3D scenes in online repositories provide new opportunities for data-driven scene understanding, editing, and synthesis. Despite the plethora of data now available online, most of it cannot be effectively used for data-driven applications because it lacks consistent segmentations, category labels, and/or functional groupings required for co-analysis. In this paper, we develop algorithms that infer such information via parsing with a probabilistic grammar learned from examples. First, given a collection of scene graphs with consistent hierarchies and labels, we train a probabilistic hierarchical grammar to represent the distributions of shapes, cardinalities, and spatial relationships of semantic objects within the collection. Then, we use the learned grammar to parse new scenes to assign them segmentations, labels, and hierarchies consistent with the collection. During experiments with these algorithms, we find that: they work effectively for scene graphs for indoor scenes commonly found online (bedrooms, classrooms, and libraries); they outperform alternative approaches that consider only shape similarities and/or spatial relationships without hierarchy; they require relatively small sets of training data; they are robust to moderate over-segmentation in the inputs; and, they can robustly transfer labels from one data set to another. As a result, the proposed algorithms can be used to provide consistent hierarchies for large collections of scenes within the same semantic class.

Original languageEnglish (US)
JournalACM transactions on graphics
Issue number6
StatePublished - Nov 19 2014

Bibliographical note

Publisher Copyright:
2014 Copyright held by the Owner/Author. Publication rights licensed to ACM.


  • Scene collections
  • Scene understanding

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Creating consistent scene graphs using a probabilistic grammar'. Together they form a unique fingerprint.

Cite this