The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.
|Original language||English (US)|
|Title of host publication||2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)|
|State||Published - Aug 23 2022|
Bibliographical noteKAUST Repository Item: Exported on 2022-09-14
Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.