Learning semantic deformation flows with 3D convolutional networks

M. Ersin Yumer; Niloy J. Mitra

doi:10.1007/978-3-319-46466-4_18

Learning semantic deformation flows with 3D convolutional networks

M. Ersin Yumer^*, Niloy J. Mitra

^*Corresponding author for this work

Computer, Electrical and Mathematical Sciences and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

34 Scopus citations

Abstract

Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self occlusions. We introduce an end-to-end solution to this tedious process using a volumetric Convolutional Neural Network (CNN) that learns deformation flows in 3D. Our network architectures take the voxelized representation of the shape and a semantic deformation intention (e.g., make more sporty) as input and generate a deformation flow at the output. We show that such deformation flows can be trivially applied to the input shape, resulting in a novel deformed version of the input without losing detail information. Our experiments show that the CNN approach achieves comparable results with state of the art methods when applied to CAD models.When applied to single frame depth scans, and partial/noisy CAD models we achieve ∼60% less error compared to the state-of-the-art.

Original language	English (US)
Title of host publication	Computer Vision - 14th European Conference, ECCV 2016, Proceedings
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Publisher	Springer Verlag
Pages	294-311
Number of pages	18
ISBN (Print)	9783319464657
DOIs	https://doi.org/10.1007/978-3-319-46466-4_18
State	Published - 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: Oct 8 2016 → Oct 16 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9910 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	10/8/16 → 10/16/16

Bibliographical note

Publisher Copyright:
© Springer International Publishing AG 2016.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-319-46466-4_18

Cite this

Yumer, M. E., & Mitra, N. J. (2016). Learning semantic deformation flows with 3D convolutional networks. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 294-311). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46466-4_18

Yumer, M. Ersin ; Mitra, Niloy J. / Learning semantic deformation flows with 3D convolutional networks. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. Springer Verlag, 2016. pp. 294-311 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{16f72657e3454751ae00df062b526e53,

title = "Learning semantic deformation flows with 3D convolutional networks",

abstract = "Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self occlusions. We introduce an end-to-end solution to this tedious process using a volumetric Convolutional Neural Network (CNN) that learns deformation flows in 3D. Our network architectures take the voxelized representation of the shape and a semantic deformation intention (e.g., make more sporty) as input and generate a deformation flow at the output. We show that such deformation flows can be trivially applied to the input shape, resulting in a novel deformed version of the input without losing detail information. Our experiments show that the CNN approach achieves comparable results with state of the art methods when applied to CAD models.When applied to single frame depth scans, and partial/noisy CAD models we achieve ∼60% less error compared to the state-of-the-art.",

author = "Yumer, {M. Ersin} and Mitra, {Niloy J.}",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

year = "2016",

doi = "10.1007/978-3-319-46466-4_18",

language = "English (US)",

isbn = "9783319464657",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "294--311",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

address = "Germany",

}

Yumer, ME & Mitra, NJ 2016, Learning semantic deformation flows with 3D convolutional networks. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9910 LNCS, Springer Verlag, pp. 294-311, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 10/8/16. https://doi.org/10.1007/978-3-319-46466-4_18

Learning semantic deformation flows with 3D convolutional networks. / Yumer, M. Ersin; Mitra, Niloy J.
Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. Springer Verlag, 2016. p. 294-311 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Learning semantic deformation flows with 3D convolutional networks

AU - Yumer, M. Ersin

AU - Mitra, Niloy J.

N1 - Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self occlusions. We introduce an end-to-end solution to this tedious process using a volumetric Convolutional Neural Network (CNN) that learns deformation flows in 3D. Our network architectures take the voxelized representation of the shape and a semantic deformation intention (e.g., make more sporty) as input and generate a deformation flow at the output. We show that such deformation flows can be trivially applied to the input shape, resulting in a novel deformed version of the input without losing detail information. Our experiments show that the CNN approach achieves comparable results with state of the art methods when applied to CAD models.When applied to single frame depth scans, and partial/noisy CAD models we achieve ∼60% less error compared to the state-of-the-art.

AB - Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self occlusions. We introduce an end-to-end solution to this tedious process using a volumetric Convolutional Neural Network (CNN) that learns deformation flows in 3D. Our network architectures take the voxelized representation of the shape and a semantic deformation intention (e.g., make more sporty) as input and generate a deformation flow at the output. We show that such deformation flows can be trivially applied to the input shape, resulting in a novel deformed version of the input without losing detail information. Our experiments show that the CNN approach achieves comparable results with state of the art methods when applied to CAD models.When applied to single frame depth scans, and partial/noisy CAD models we achieve ∼60% less error compared to the state-of-the-art.

UR - http://www.scopus.com/inward/record.url?scp=84990043610&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46466-4_18

DO - 10.1007/978-3-319-46466-4_18

M3 - Conference contribution

AN - SCOPUS:84990043610

SN - 9783319464657

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 294

EP - 311

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

PB - Springer Verlag

T2 - 14th European Conference on Computer Vision, ECCV 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Yumer ME, Mitra NJ. Learning semantic deformation flows with 3D convolutional networks. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 294-311. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-46466-4_18

Learning semantic deformation flows with 3D convolutional networks

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this