LC-NAS: Latency Constrained Neural Architecture Search for Point Cloud Networks

Point cloud architecture design has become a crucial problem for deep learning in 3D. Several efforts have been made to manually design architectures targeting high accuracy in point cloud tasks such as classification, segmentation, and detection. Recent progress in automatic Neural Architecture Search (NAS) minimizes the human effort in network design and optimizes architectures for high performance. However, those efforts fail to consider crucial factors such as latency during inference, which is of high importance in time-critical and hardware-bounded applications like self-driving cars, robot navigation, and mobile applications. In this paper, we introduce a new NAS framework, dubbed LC-NAS, that searches for point cloud architectures constrained to a target latency. We implement a novel latency constraint formulation for the trade-off between accuracy and latency in our architecture search. Contrary to previous works, our latency loss enables us to find the best architecture with latency near a specific target value, which is crucial when the end task is to be deployed in a limited hardware setting. Extensive experiments show that LC-NAS is able to find state-of-the-art architectures for point cloud classification in ModelNet40 with a minimal computational cost. We also show how our searched architectures achieve any desired latency with a reasonably low drop in accuracy. Finally, we show how our searched architectures easily transfer to the part segmentation task on PartNet, where we achieve state-of-the-art results with significantly lower latency.