Although Resistive RAMs can support highly efficient matrix-vector multiplication, which is very useful for machine learning and other applications, the non-ideal behavior of hardware such as stuck-at fault and IR drop is an important concern in making ReRAM crossbar array-based deep learning accelerators. Previous work has addressed the nonideality problem through either redundancy in hardware, which requires a permanent increase of hardware cost, or software retraining, which may be even more costly or unacceptable due to its need for a training dataset as well as high computation overhead. In this paper we propose a very light-weight method that can be applied on top of existing hardware or software solutions. Our method, called FPT (Forward-Parameter Tuning), takes advantage of a certain statistical property existing in the activation data of neural network layers, and can mitigate the impact of mild nonidealities in ReRAM crossbar arrays for deep learning applications without using any hardware, a dataset, or gradientbased training. Our experimental results using MNIST, CIFAR-10, CIFAR-100, and ImageNet datasets in binary and multibit networks demonstrate that our technique is very effective, both alone and together with previous methods, up to 20rate, which is higher than even some of the previous remapping methods. We also evaluate our method in the presence of other nonidealities such as variability and IR drop. Further, we provide an analysis based on the concept of effective fault rate, which not only demonstrates that effective fault rate can be a useful tool to predict the accuracy of faulty RCA-based neural networks, but also explains why mitigating the SAF problem is more difficult with multi-bit neural networks.
|Original language||English (US)|
|Number of pages||1|
|Journal||IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems|
|State||Published - Nov 15 2022|
Bibliographical noteKAUST Repository Item: Exported on 2022-11-18
Acknowledgements: This work was supported by IITP grants (No. 2020-0-01336, Artificial Intelligence Graduate School Program (UNIST) and IITP-2021-0-02052, ITRC support program) and NRF grant (No. 2020R1A2C2015066) funded by MSIT of Korea, and by Free Innovative Research Fund of UNIST (1.170067.01). The EDA tool was supported by the IC Design Education Center (IDEC), Korea.
ASJC Scopus subject areas
- Computer Graphics and Computer-Aided Design
- Electrical and Electronic Engineering