Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline

Guocheng Qian, Yuanhao Wang, Jinjin Gu, Chao Dong, Wolfgang Heidrich, Bernard Ghanem, Jimmy S. Ren

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

Imaging is usually a mixture problem of incomplete color sampling, noise degradation, and limited resolution. This mixture problem is typically solved by a sequential solution that applies demosaicing (DM), denoising (DN), and super-resolution (SR) sequentially in a fixed and predefined pipeline (execution order of tasks), DM→DN→SR. The most recent work on image processing focuses on developing more sophisticated architectures to achieve higher image quality. Little attention has been paid to the design of the pipeline, and it is still not clear how significant the pipeline is to image quality. In this work, we comprehensively study the effects of pipelines on the mixture problem of learning-based DN, DM, and SR, in both sequential and joint solutions. On the one hand, in sequential solutions, we find that the pipeline has a non-trivial effect on the resulted image quality. Our suggested pipeline DN→SR→DM yields consistently better performance than other sequential pipelines in various experimental settings and benchmarks. On the other hand, in joint solutions, we propose an end-to-end Trinity Pixel Enhancement NETwork (TENet) that achieves the state-of-the-art performance for the mixture problem. We further present a novel and simple method that can integrate a certain pipeline into a given end-to-end network by providing intermediate supervision using a detachable head. Extensive experiments show that an end-to-end network with the proposed pipeline can attain only a consistent but insignificant improvement. Our work indicates that the investigation of pipelines is applicable in sequential solutions, but is not very necessary in end-to-end networks.
Original languageEnglish (US)
Title of host publication2022 IEEE International Conference on Computational Photography (ICCP)
PublisherIEEE
ISBN (Print)978-1-6654-5852-8
DOIs
StatePublished - Sep 26 2022

Bibliographical note

KAUST Repository Item: Exported on 2022-12-14
Acknowledgements: The authors thank the reviewers of ICCP 2022 for valuable suggestions and Dr. Silvio Giancola for proofreading the rebuttals. This work was supported by the KAUST Office of Sponsored Research (OSR) through the Visual Computing Center (VCC) funding.

Fingerprint

Dive into the research topics of 'Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline'. Together they form a unique fingerprint.

Cite this