Deep adaptive LiDAR: End-to-end optimization of sampling and depth completion at low sampling rates

Alexander W. Bergman; David B. Lindell; Gordon Wetzstein

doi:10.1109/ICCP48838.2020.9105252

Deep adaptive LiDAR: End-to-end optimization of sampling and depth completion at low sampling rates

Alexander W. Bergman, David B. Lindell, Gordon Wetzstein

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

28 Scopus citations

Abstract

Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.

Original language	English (US)
Title of host publication	2020 IEEE International Conference on Computational Photography (ICCP)
Publisher	IEEE
ISBN (Print)	9781728152301
DOIs	https://doi.org/10.1109/ICCP48838.2020.9105252
State	Published - Jun 2 2020
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-30
Acknowledgements: A.W.B. and D.B.L. are supported by a Stanford Graduate Fellowship in Science and Engineering. This project was supported by a Terman Faculty Fellowship, a Sloan Fellowship, a NSF CAREER Award (IIS 1553333), the DARPA REVEAL program, the ARO (ECASE-Army Award W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Access to Document

10.1109/ICCP48838.2020.9105252

Cite this

@inproceedings{3654041a3a584810b1ec9cb90e5cd97f,

title = "Deep adaptive LiDAR: End-to-end optimization of sampling and depth completion at low sampling rates",

abstract = "Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.",

author = "Bergman, {Alexander W.} and Lindell, {David B.} and Gordon Wetzstein",

note = "KAUST Repository Item: Exported on 2022-06-30 Acknowledgements: A.W.B. and D.B.L. are supported by a Stanford Graduate Fellowship in Science and Engineering. This project was supported by a Terman Faculty Fellowship, a Sloan Fellowship, a NSF CAREER Award (IIS 1553333), the DARPA REVEAL program, the ARO (ECASE-Army Award W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2020",

month = jun,

day = "2",

doi = "10.1109/ICCP48838.2020.9105252",

language = "English (US)",

isbn = "9781728152301",

booktitle = "2020 IEEE International Conference on Computational Photography (ICCP)",

publisher = "IEEE",

}

TY - GEN

T1 - Deep adaptive LiDAR: End-to-end optimization of sampling and depth completion at low sampling rates

AU - Bergman, Alexander W.

AU - Lindell, David B.

AU - Wetzstein, Gordon

N1 - KAUST Repository Item: Exported on 2022-06-30 Acknowledgements: A.W.B. and D.B.L. are supported by a Stanford Graduate Fellowship in Science and Engineering. This project was supported by a Terman Faculty Fellowship, a Sloan Fellowship, a NSF CAREER Award (IIS 1553333), the DARPA REVEAL program, the ARO (ECASE-Army Award W911NF-19-1-0120), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2020/6/2

Y1 - 2020/6/2

N2 - Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.

AB - Current LiDAR systems are limited in their ability to capture dense 3D point clouds. To overcome this challenge, deep learning-based depth completion algorithms have been developed to inpaint missing depth guided by an RGB image. However, these methods fail for low sampling rates. Here, we propose an adaptive sampling scheme for LiDAR systems that demonstrates state-of-the-art performance for depth completion at low sampling rates. Our system is fully differentiable, allowing the sparse depth sampling and the depth inpainting components to be trained end-to-end with an upstream task.

UR - http://hdl.handle.net/10754/679495

UR - https://ieeexplore.ieee.org/document/9105252/

UR - http://www.scopus.com/inward/record.url?scp=85086634734&partnerID=8YFLogxK

U2 - 10.1109/ICCP48838.2020.9105252

DO - 10.1109/ICCP48838.2020.9105252

M3 - Conference contribution

SN - 9781728152301

BT - 2020 IEEE International Conference on Computational Photography (ICCP)

PB - IEEE

ER -

Deep adaptive LiDAR: End-to-end optimization of sampling and depth completion at low sampling rates

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this