Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images

David Dov, Shahar Z. Kovalsky, Serge Assaad, Jonathan Cohen, Danielle Elliott Range, Avani A. Pendse, Ricardo Henao, Lawrence Carin

Research output: Contribution to journalArticlepeer-review

43 Scopus citations


We consider machine-learning-based thyroid-malignancy prediction from cytopathology whole-slide images (WSI). Multiple instance learning (MIL) approaches, typically used for the analysis of WSIs, divide the image (bag) into patches (instances), which are used to predict a single bag-level label. These approaches perform poorly in cytopathology slides due to a unique bag structure: sparsely located informative instances with varying characteristics of abnormality. We address these challenges by considering multiple types of labels: bag-level malignancy and ordered diagnostic scores, as well as instance-level informativeness and abnormality labels. We study their contribution beyond the MIL setting by proposing a maximum likelihood estimation (MLE) framework, from which we derive a two-stage deep-learning-based algorithm. The algorithm identifies informative instances and assigns them local malignancy scores that are incorporated into a global malignancy prediction. We derive a lower bound of the MLE, leading to an improved training strategy based on weak supervision, that we motivate through statistical analysis. The lower bound further allows us to extend the proposed algorithm to simultaneously predict multiple bag and instance-level labels from a single output of a neural network. Experimental results demonstrate that the proposed algorithm provides competitive performance compared to several competing methods, achieves (expert) human-level performance, and allows augmentation of human decisions.
Original languageEnglish (US)
JournalMedical Image Analysis
StatePublished - Jan 1 2021
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2021-02-09


Dive into the research topics of 'Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images'. Together they form a unique fingerprint.

Cite this