Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors

Zhongxiao Li; Yuwei Cong; Xin Chen; Jiping Qi; Jingxian Sun; Tao Yan; He Yang; Junsi Liu; Enzhou Lu; Lixiang Wang; Jiafeng Li; Hong Hu; Cheng Zhang; Quan Yang; Jiawei Yao; Penglei Yao; Qiuyi Jiang; Wenwu Liu; Jiangning Song; Lawrence Carin; Yupeng Chen; SHIGUANG ZHAO; Xin Gao

doi:10.1016/j.isci.2022.105872

Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors

Zhongxiao Li, Yuwei Cong, Xin Chen, Jiping Qi, Jingxian Sun, Tao Yan, He Yang, Junsi Liu, Enzhou Lu, Lixiang Wang, Jiafeng Li, Hong Hu, Cheng Zhang, Quan Yang, Jiawei Yao, Penglei Yao, Qiuyi Jiang, Wenwu Liu, Jiangning Song, Lawrence CarinYupeng Chen, SHIGUANG ZHAO, Xin Gao

Research output: Contribution to journal › Article › peer-review

28 Scopus citations

Abstract

Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region-of-interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly-supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.

Original language	English (US)
Pages (from-to)	105872
Journal	iScience
DOIs	https://doi.org/10.1016/j.isci.2022.105872
State	Published - Dec 24 2022

Bibliographical note

KAUST Repository Item: Exported on 2022-12-26
Acknowledged KAUST grant number(s): BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, REI/1/4742-01-01, URF/1/4098-01-01, URF/1/4352-01-01
Acknowledgements: This work was supported by Office of Research Administration (ORA) at KAUST under award numbers BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, URF/1/4098-01-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, URF/1/4352-01-01, and REI/1/4742-01-01. Figure 1 was created by Heno Hwang, scientific illustrator at King Abdullah University of Science and Technology (KAUST).

Access to Document

10.1016/j.isci.2022.105872

https://repository.kaust.edu.sa/bitstream/10754/686615/1/1-s2.0-S2589004222021459-main_removed.pdf

Cite this

@article{50e3d22345da455ebc5c74e4aa090fca,

title = "Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors",

abstract = "Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists{\textquoteright} annotation of region-of-interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly-supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.",

author = "Zhongxiao Li and Yuwei Cong and Xin Chen and Jiping Qi and Jingxian Sun and Tao Yan and He Yang and Junsi Liu and Enzhou Lu and Lixiang Wang and Jiafeng Li and Hong Hu and Cheng Zhang and Quan Yang and Jiawei Yao and Penglei Yao and Qiuyi Jiang and Wenwu Liu and Jiangning Song and Lawrence Carin and Yupeng Chen and SHIGUANG ZHAO and Xin Gao",

note = "KAUST Repository Item: Exported on 2022-12-26 Acknowledged KAUST grant number(s): BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, REI/1/4742-01-01, URF/1/4098-01-01, URF/1/4352-01-01 Acknowledgements: This work was supported by Office of Research Administration (ORA) at KAUST under award numbers BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, URF/1/4098-01-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, URF/1/4352-01-01, and REI/1/4742-01-01. Figure 1 was created by Heno Hwang, scientific illustrator at King Abdullah University of Science and Technology (KAUST).",

year = "2022",

month = dec,

day = "24",

doi = "10.1016/j.isci.2022.105872",

language = "English (US)",

pages = "105872",

journal = "iScience",

issn = "2589-0042",

publisher = "Elsevier Inc.",

}

Li, Z, Cong, Y, Chen, X, Qi, J, Sun, J, Yan, T, Yang, H, Liu, J, Lu, E, Wang, L, Li, J, Hu, H, Zhang, C, Yang, Q, Yao, J, Yao, P, Jiang, Q, Liu, W, Song, J, Carin, L, Chen, Y, ZHAO, SHIGUANG & Gao, X 2022, 'Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors', iScience, pp. 105872. https://doi.org/10.1016/j.isci.2022.105872

TY - JOUR

T1 - Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors

AU - Li, Zhongxiao

AU - Cong, Yuwei

AU - Chen, Xin

AU - Qi, Jiping

AU - Sun, Jingxian

AU - Yan, Tao

AU - Yang, He

AU - Liu, Junsi

AU - Lu, Enzhou

AU - Wang, Lixiang

AU - Li, Jiafeng

AU - Hu, Hong

AU - Zhang, Cheng

AU - Yang, Quan

AU - Yao, Jiawei

AU - Yao, Penglei

AU - Jiang, Qiuyi

AU - Liu, Wenwu

AU - Song, Jiangning

AU - Carin, Lawrence

AU - Chen, Yupeng

AU - ZHAO, SHIGUANG

AU - Gao, Xin

N1 - KAUST Repository Item: Exported on 2022-12-26 Acknowledged KAUST grant number(s): BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, REI/1/4742-01-01, URF/1/4098-01-01, URF/1/4352-01-01 Acknowledgements: This work was supported by Office of Research Administration (ORA) at KAUST under award numbers BAS/1/1624-01, FCC/1/1976-18-01, FCC/1/1976-23-01, FCC/1/1976-25-01, FCC/1/1976-26-01, URF/1/4098-01-01, REI/1/0018-01-01, REI/1/4216-01-01, REI/1/4437-01-01, REI/1/4473-01-01, URF/1/4352-01-01, and REI/1/4742-01-01. Figure 1 was created by Heno Hwang, scientific illustrator at King Abdullah University of Science and Technology (KAUST).

PY - 2022/12/24

Y1 - 2022/12/24

N2 - Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region-of-interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly-supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.

AB - Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region-of-interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly-supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.

UR - http://hdl.handle.net/10754/686615

UR - https://linkinghub.elsevier.com/retrieve/pii/S2589004222021459

U2 - 10.1016/j.isci.2022.105872

DO - 10.1016/j.isci.2022.105872

M3 - Article

C2 - 36647383

SN - 2589-0042

SP - 105872

JO - iScience

JF - iScience

ER -

Vision Transformer-based Weakly-Supervised Histopathological Image Analysis of Primary Brain Tumors

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this