Abstract
State-of-the-art face super-resolution methods leverage deep convolutional neural networks to learn a mapping between low-resolution (LR) facial patterns and their corresponding high-resolution (HR) counterparts by exploring local appearance information. However, most of these methods do not account for facial structure and suffer from degradations due to large pose variations and misalignments. In this paper, we propose a method that explicitly incorporates structural information of faces into the face super-resolution process by using a multi-task convolutional neural network (CNN). Our CNN has two branches: one for super-resolving face images and the other branch for predicting salient regions of a face coined facial component heatmaps. These heatmaps encourage the upsampling stream to generate super-resolved faces with higher-quality details. Our method not only uses low-level information (i.e., intensity similarity), but also middle-level information (i.e., face structure) to further explore spatial constraints of facial components from LR inputs images. Therefore, we are able to super-resolve very small unaligned face images (16×16pixels) with a large upscaling factor of 8 ×, while preserving face structure. Extensive experiments demonstrate that our network achieves superior face hallucination results and outperforms the state-of-the-art.
Original language | English (US) |
---|---|
Title of host publication | Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings |
Editors | Martial Hebert, Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss |
Publisher | Springer Verlag |
Pages | 219-235 |
Number of pages | 17 |
ISBN (Print) | 9783030012397 |
DOIs | |
State | Published - 2018 |
Event | 15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: Sep 8 2018 → Sep 14 2018 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 11213 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 15th European Conference on Computer Vision, ECCV 2018 |
---|---|
Country/Territory | Germany |
City | Munich |
Period | 09/8/18 → 09/14/18 |
Bibliographical note
Publisher Copyright:© Springer Nature Switzerland AG 2018.
Keywords
- Face
- Facial component localization
- Hallucination
- Multi-task neural networks
- Super-resolution
ASJC Scopus subject areas
- Theoretical Computer Science
- General Computer Science