Non-line-of-sight imaging with partial occluders and surface normals

Felix Heide; Matthew O'Toole; Kai Zang; David B. Lindell; Steven Diamond; Gordon Wetzstein

doi:10.1145/3269977

Non-line-of-sight imaging with partial occluders and surface normals

Felix Heide, Matthew O'Toole, Kai Zang, David B. Lindell, Steven Diamond, Gordon Wetzstein

Research output: Contribution to journal › Article › peer-review

95 Scopus citations

Abstract

Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.

Original language	English (US)
Pages (from-to)	1-10
Number of pages	10
Journal	ACM Transactions on Graphics
Volume	38
Issue number	3
DOIs	https://doi.org/10.1145/3269977
State	Published - May 9 2019
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-10
Acknowledgements: D. B. L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G. W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant. The authors would like to thank James Harris for fruitful discussions. D.B.L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G.W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Access to Document

10.1145/3269977

Cite this

@article{cbcf46e19e3e428d93eae8c6402130ce,

title = "Non-line-of-sight imaging with partial occluders and surface normals",

abstract = "Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.",

author = "Felix Heide and Matthew O'Toole and Kai Zang and Lindell, {David B.} and Steven Diamond and Gordon Wetzstein",

note = "KAUST Repository Item: Exported on 2022-06-10 Acknowledgements: D. B. L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G. W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant. The authors would like to thank James Harris for fruitful discussions. D.B.L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G.W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2019",

month = may,

day = "9",

doi = "10.1145/3269977",

language = "English (US)",

volume = "38",

pages = "1--10",

journal = "ACM Transactions on Graphics",

issn = "1557-7368",

publisher = "Association for Computing Machinery",

number = "3",

}

TY - JOUR

T1 - Non-line-of-sight imaging with partial occluders and surface normals

AU - Heide, Felix

AU - O'Toole, Matthew

AU - Zang, Kai

AU - Lindell, David B.

AU - Diamond, Steven

AU - Wetzstein, Gordon

N1 - KAUST Repository Item: Exported on 2022-06-10 Acknowledgements: D. B. L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G. W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant. The authors would like to thank James Harris for fruitful discussions. D.B.L. is supported by a Stanford Graduate Fellowship in Science and Engineering. G.W. is supported by a Terman Faculty Fellowship and a Sloan Fellowship. Additional funding was generously provided by the National Science Foundation (CAREER Award IIS 1553333), the DARPA REVEAL program, the ARO (Grant W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2019/5/9

Y1 - 2019/5/9

N2 - Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.

AB - Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.

UR - http://hdl.handle.net/10754/678851

UR - https://dl.acm.org/doi/10.1145/3269977

UR - http://www.scopus.com/inward/record.url?scp=85065768416&partnerID=8YFLogxK

U2 - 10.1145/3269977

DO - 10.1145/3269977

M3 - Article

SN - 1557-7368

VL - 38

SP - 1

EP - 10

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

IS - 3

ER -

Non-line-of-sight imaging with partial occluders and surface normals

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this