Learning from Weak and Noisy Labels for Semantic Segmentation

Zhiwu Lu, Zhenyong Fu, Tao Xiang, Peng Han, Liwei Wang, Xin Gao

Research output: Contribution to journalArticlepeer-review

77 Scopus citations


A weakly supervised semantic segmentation (WSSS) method aims to learn a segmentation model from weak (image-level) as opposed to strong (pixel-level) labels. By avoiding the tedious pixel-level annotation process, it can exploit the unlimited supply of user-tagged images from media-sharing sites such as Flickr for large scale applications. However, these ‘free’ tags/labels are often noisy and few existing works address the problem of learning with both weak and noisy labels. In this work, we cast the WSSS problem into a label noise reduction problem. Specifically, after segmenting each image into a set of superpixels, the weak and potentially noisy image-level labels are propagated to the superpixel level resulting in highly noisy labels; the key to semantic segmentation is thus to identify and correct the superpixel noisy labels. To this end, a novel L1-optimisation based sparse learning model is formulated to directly and explicitly detect noisy labels. To solve the L1-optimisation problem, we further develop an efficient learning algorithm by introducing an intermediate labelling variable. Extensive experiments on three benchmark datasets show that our method yields state-of-the-art results given noise-free labels, whilst significantly outperforming the existing methods when the weak labels are also noisy.
Original languageEnglish (US)
Pages (from-to)486-500
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number3
StatePublished - Apr 9 2016

Bibliographical note

KAUST Repository Item: Exported on 2020-10-01
Acknowledgements: This work was partially supported by National Natural Science
Foundation of China (61573363 and 61573026), 973
Program of China (2014CB340403 and 2015CB352502),
the Fundamental Research Funds for the Central Universities
and the Research Funds of Renmin University of China
(15XNLQ01), IBM Global SUR Award Program, European
Research Council FP7 Project SUNNY (313243), and the
funding from KAUST.


Dive into the research topics of 'Learning from Weak and Noisy Labels for Semantic Segmentation'. Together they form a unique fingerprint.

Cite this