When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Guocheng Qian; Xuanyang Zhang; Guohao Li; Chen Zhao; Yukang Chen; Xiangyu Zhang; Bernard Ghanem; Jian Sun

doi:10.1109/CVPRW56347.2022.00314

When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Guocheng Qian, Xuanyang Zhang, Guohao Li, Chen Zhao, Yukang Chen, Xiangyu Zhang, Bernard Ghanem, Jian Sun

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

Original language	English (US)
Title of host publication	2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Publisher	IEEE
ISBN (Print)	978-1-6654-8740-5
DOIs	https://doi.org/10.1109/CVPRW56347.2022.00314
State	Published - Aug 23 2022

Bibliographical note

KAUST Repository Item: Exported on 2022-09-14
Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.

Access to Document

10.1109/CVPRW56347.2022.00314

https://repository.kaust.edu.sa/bitstream/10754/676640/5/CVPR2022W_TNAS.pdf

Cite this

@inproceedings{981d23937eff4f6cb7f46dc9cd662295,

title = "When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search",

abstract = "The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.",

author = "Guocheng Qian and Xuanyang Zhang and Guohao Li and Chen Zhao and Yukang Chen and Xiangyu Zhang and Bernard Ghanem and Jian Sun",

note = "KAUST Repository Item: Exported on 2022-09-14 Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.",

year = "2022",

month = aug,

day = "23",

doi = "10.1109/CVPRW56347.2022.00314",

language = "English (US)",

isbn = "978-1-6654-8740-5",

booktitle = "2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)",

publisher = "IEEE",

}

TY - GEN

T1 - When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

AU - Qian, Guocheng

AU - Zhang, Xuanyang

AU - Li, Guohao

AU - Zhao, Chen

AU - Chen, Yukang

AU - Zhang, Xiangyu

AU - Ghanem, Bernard

AU - Sun, Jian

N1 - KAUST Repository Item: Exported on 2022-09-14 Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.

PY - 2022/8/23

Y1 - 2022/8/23

N2 - The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

AB - The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

UR - http://hdl.handle.net/10754/676640

UR - https://ieeexplore.ieee.org/document/9857078/

U2 - 10.1109/CVPRW56347.2022.00314

DO - 10.1109/CVPRW56347.2022.00314

M3 - Conference contribution

SN - 978-1-6654-8740-5

BT - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

PB - IEEE

ER -

When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this