Fast and Low-Cost Mitigation of ReRAM Variability for Deep Learning Applications

Sugil Lee, Mohammed Fouda, Jongeun Lee, Ahmed Eltawil, Fadi Kurdahi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

To overcome the programming variability (PV) of ReRAM crossbar arrays (RCAs), the most common method is program-verify, which, however, has high energy and latency overhead. In this paper we propose a very fast and low-cost method to mitigate the effect of PV and other variability for RCA-based DNN (Deep Neural Network) accelerators. Leveraging the statistical properties of DNN output, our method called Online Batch-Norm Correction (OBNC) can compensate for the effect of programming and other variability on RCA output without using on-chip training or an iterative procedure, and is thus very fast. Also our method does not require a nonideality model or a training dataset, hence very easy to apply. Our experimental results using ternary neural networks with binary and 4-bit activations demonstrate that our OBNC can recover the baseline performance in many variability settings and that our method outperforms a previously known method (VCAM) by large margins when input distribution is asymmetric or activation is multi-bit.
Original languageEnglish (US)
Title of host publication2021 IEEE 39th International Conference on Computer Design (ICCD)
PublisherIEEE
Pages269-276
Number of pages8
ISBN (Print)9781665432191
DOIs
StatePublished - Oct 2021

Bibliographical note

KAUST Repository Item: Exported on 2022-03-21
Acknowledgements: This work was supported by NRF grant (No. 2020R1A2C2015066) and IITP grant (No. 2020-0-01336, Artificial Intelligence Graduate School Program) funded by MSIT of Korea, and by Free Innovative Research Fund of UNIST (1.170067.01). The EDA tool was supported by the IC Design Education Center (IDEC), Korea.

Fingerprint

Dive into the research topics of 'Fast and Low-Cost Mitigation of ReRAM Variability for Deep Learning Applications'. Together they form a unique fingerprint.

Cite this