We present a room layout estimation framework that jointly learns wide baseline panorama registration and layout estimation given a pair of 360° panoramas. To effectively tackle the wide baseline registration problem, we introduce a novel end-to-end supervised Geometry-aware Panorama Registration Network or GPR-Net that exploits the layout geometry and computes fine-grained correspondences on the layout boundary, instead of the global pixel-space. GPR-Net consists of two main parts. The geometry transformer learns a set of 1D horizon features sampled on the panorama. These 1D feature maps encode geometric cues describing the ceiling-wall and floor-wall layout boundaries, and the correspondence and co-visibility between layout boundaries. These learned geometric cues are further used for direct regression of relative pose (translation and rotation) with a pose transformer. The final layout is then obtained by registering the two layouts using the estimated pose and taking the union of the two individual layouts derived from the estimated layout boundary maps. Experimental results indicate that our method achieves state-of-the-art performance in both panorama registration and layout estimation on a large-scale indoor panorama dataset ZInD . Our code is available online1.
|Title of host publication
|Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
|IEEE Computer Society
|Number of pages
|Published - 2023
|2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023 - Vancouver, Canada
Duration: Jun 18 2023 → Jun 22 2023
|IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
|2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
|06/18/23 → 06/22/23
Bibliographical notePublisher Copyright:
© 2023 IEEE.
ASJC Scopus subject areas
- Computer Vision and Pattern Recognition
- Electrical and Electronic Engineering