Abstract
This paper proposes a novel approach for rendering a pre-trained Neural Radiance Field (NeRF) in real-time on resource-constrained devices. We introduce Re-ReND, a method enabling Real-time Rendering of NeRFs across Devices. Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines. The proposed method distills the NeRF by extracting the learned density into a mesh, while the learned color information is factorized into a set of matrices that represent the scene's light field. Factorization implies the field is queried via inexpensive MLP-free matrix multiplications, while using a light field allows rendering a pixel by querying the field a single time - as opposed to hundreds of queries when employing a radiance field. Since the proposed representation can be implemented using a fragment shader, it can be directly integrated with standard rasterization frameworks. Our flexible implementation can render a NeRF in real-time with low memory requirements and on a wide range of resource-constrained devices, including mobiles and AR/VR headsets. Notably, we find that Re-ReND can achieve over a 2.6-fold increase in rendering speed versus the state-of-the-art without perceptible losses in quality.
Original language | English (US) |
---|---|
Title of host publication | Proceedings - 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 3609-3618 |
Number of pages | 10 |
ISBN (Electronic) | 9798350307184 |
DOIs | |
State | Published - 2023 |
Event | 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023 - Paris, France Duration: Oct 2 2023 → Oct 6 2023 |
Publication series
Name | Proceedings of the IEEE International Conference on Computer Vision |
---|---|
ISSN (Print) | 1550-5499 |
Conference
Conference | 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023 |
---|---|
Country/Territory | France |
City | Paris |
Period | 10/2/23 → 10/6/23 |
Bibliographical note
Publisher Copyright:© 2023 IEEE.
ASJC Scopus subject areas
- Software
- Computer Vision and Pattern Recognition