dO: A differentiable engine for Deep Lens design of computational imaging systems

Congli Wang, Chen Ni, Wolfgang Heidrich

Research output: Contribution to journalArticlepeer-review


Computational imaging systems algorithmically post-process acquisition images either to reveal physical quantities of interest or to increase image quality, e.g., deblurring. Designing a computational imaging system requires co-design of optics and algorithms, and recently Deep Lens systems have been proposed in which both components are end-to-end designed using data-driven end-to-end training. However, progress on this exciting concept has so far been hampered by the lack of differentiable forward simulations for complex optical design spaces. Here, we introduce dO (DiffOptics) to provide derivative insights into the design pipeline to chain variable parameters and their gradients to an error metric through differential ray tracing. However, straightforward back-propagation of many millions of rays requires unaffordable device memory, and is not resolved by prior works. dO alleviates this issue using two customized memory-efficient techniques: differentiable ray surface intersection and adjoint back-propagation. Broad application examples demonstrate the versatility and flexibility of dO, including classical lens designs in asphere, double-Gauss, and freeform, reverse engineering for metrology, and joint designs of optics-network in computational imaging applications. We believe dO enables a radically new approach to computational imaging system designs and relevant research domains.
Original languageEnglish (US)
StatePublished - 2021

Bibliographical note

KAUST Repository Item: Exported on 2022-01-06
Acknowledgements: This work was supported by baseline funding of the King Abdullah University of Science and Technology


Dive into the research topics of 'dO: A differentiable engine for Deep Lens design of computational imaging systems'. Together they form a unique fingerprint.

Cite this