Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification

Julie Chang*, Vincent Sitzmann, Xiong Dun, Wolfgang Heidrich, Gordon Wetzstein

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

323 Scopus citations

Abstract

Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.

Original languageEnglish (US)
Article number12324
JournalScientific Reports
Volume8
Issue number1
DOIs
StatePublished - Dec 1 2018

Bibliographical note

Publisher Copyright:
© 2018, The Author(s).

ASJC Scopus subject areas

  • General

Fingerprint

Dive into the research topics of 'Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification'. Together they form a unique fingerprint.

Cite this