Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects along a Single Optical Path

Christopher A. Metzler; David B. Lindell; Gordon Wetzstein

doi:10.1109/TCI.2020.3046472

Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects along a Single Optical Path

Christopher A. Metzler, David B. Lindell, Gordon Wetzstein

Research output: Contribution to journal › Article › peer-review

29 Scopus citations

Abstract

Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.

Original language	English (US)
Pages (from-to)	1-12
Number of pages	12
Journal	IEEE Transactions on Computational Imaging
Volume	7
DOIs	https://doi.org/10.1109/TCI.2020.3046472
State	Published - Dec 22 2020
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-14
Acknowledgements: The work of C. A. Metzler was supported by an appointment to the Intelligence Community Postdoctoral Research Fellowship Program at Stanford University administered by Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between the U.S. Department of Energy and the Office of the Director of National Intelligence (ODN). The work of D. B. Lindell was supported by a Stanford Graduate Fellowship. The work of G. Wetzstein was supported in part by the NSF CAREER Award (IIS 1553333), in part by the Sloan Fellowship, in part by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and in part by the PECASE by the ARL.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Access to Document

10.1109/TCI.2020.3046472

Cite this

@article{412d63b7581f42a292936163e41e6c1c,

title = "Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects along a Single Optical Path",

abstract = "Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.",

author = "Metzler, {Christopher A.} and Lindell, {David B.} and Gordon Wetzstein",

note = "KAUST Repository Item: Exported on 2022-06-14 Acknowledgements: The work of C. A. Metzler was supported by an appointment to the Intelligence Community Postdoctoral Research Fellowship Program at Stanford University administered by Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between the U.S. Department of Energy and the Office of the Director of National Intelligence (ODN). The work of D. B. Lindell was supported by a Stanford Graduate Fellowship. The work of G. Wetzstein was supported in part by the NSF CAREER Award (IIS 1553333), in part by the Sloan Fellowship, in part by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and in part by the PECASE by the ARL. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2020",

month = dec,

day = "22",

doi = "10.1109/TCI.2020.3046472",

language = "English (US)",

volume = "7",

pages = "1--12",

journal = "IEEE Transactions on Computational Imaging",

issn = "2333-9403",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects along a Single Optical Path

AU - Metzler, Christopher A.

AU - Lindell, David B.

AU - Wetzstein, Gordon

N1 - KAUST Repository Item: Exported on 2022-06-14 Acknowledgements: The work of C. A. Metzler was supported by an appointment to the Intelligence Community Postdoctoral Research Fellowship Program at Stanford University administered by Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between the U.S. Department of Energy and the Office of the Director of National Intelligence (ODN). The work of D. B. Lindell was supported by a Stanford Graduate Fellowship. The work of G. Wetzstein was supported in part by the NSF CAREER Award (IIS 1553333), in part by the Sloan Fellowship, in part by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and in part by the PECASE by the ARL. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2020/12/22

Y1 - 2020/12/22

N2 - Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.

AB - Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.

UR - http://hdl.handle.net/10754/679005

UR - https://ieeexplore.ieee.org/document/9302876/

UR - http://www.scopus.com/inward/record.url?scp=85098779480&partnerID=8YFLogxK

U2 - 10.1109/TCI.2020.3046472

DO - 10.1109/TCI.2020.3046472

M3 - Article

SN - 2333-9403

VL - 7

SP - 1

EP - 12

JO - IEEE Transactions on Computational Imaging

JF - IEEE Transactions on Computational Imaging

ER -

Keyhole Imaging:Non-Line-of-Sight Imaging and Tracking of Moving Objects along a Single Optical Path

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this