Hardware-Friendly Lightweight Convolutional Neural Network Derivation at The Edge

Convolutional neural networks(CNNs) have demonstrated remarkable capability and scalability in a variety of vision-related tasks. Due to privacy and latency constraints, in some scenarios, the CNNs are deployed on-site where power supply, computation power, and memory capacity are limited. These constraints hinder the traditional training or modification of CNN models, which typically involves network-scale backpropagation of the gradients. In this work, we proposed a framework enabling the derivation of lightweight models from the original model at the edge only utilizing hardware-friendly operations. In the proposed framework, all models are binary quantized and the gradients are obtained by layer-wise decision boundary matching. Hence, the whole flow can be executed with bit-wise and fixed-point arithmetic operations without network-scale gradient backpropagations. The derived model serves as a viable alternative to the original, in scenarios where the accuracy requirements are less stringent, delivering enhanced efficiencies in power and memory consumption. We validate the framework on digit recognition tasks, obtaining a better accuracy than naively deploying the same lightweight model. Furthermore, an FPGA demonstration of our framework achieved a throughput of 2.2 TOPS/s, underscoring its practical applicability.