2022 3.12 - ADOP: Approximate Differentiable One-Pixel Point Rendering/ClipID:38155 previous clip next clip

The automatic subtitles generated using Whisper Open AI in this video player (and in the Multistream video player) are provided for convenience and accessibility purposes. However, please note that accuracy and interpretation may vary. For more information, please refer to the FAQs (Paragraph 14).
Recording date 2021-11-17

Via

Free

Language

English

Organisational Unit

Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)

Producer

Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)

Format

lecture

Paper on arXiv: https://arxiv.org/abs/2110.06635

Abstract: We present a novel point-based, differentiable neural rendering pipeline for scene refinement and novel view synthesis. The input are an initial estimate of the point cloud and the camera parameters. The output are synthesized images from arbitrary camera poses. The point cloud rendering is performed by a differentiable renderer using multi-resolution one-pixel point rasterization. Spatial gradients of the discrete rasterization are approximated by the novel concept of ghost geometry. After rendering, the neural image pyramid is passed through a deep neural network for shading calculations and hole-filling. A differentiable, physically-based tonemapper then converts the intermediate output to the target image. Since all stages of the pipeline are differentiable, we optimize all of the scene's parameters i.e. camera model, camera pose, point position, point color, environment map, rendering network weights, vignetting, camera response function, per image exposure, and per image white balance. We show that our system is able to synthesize sharper and more consistent novel views than existing approaches because the initial reconstruction is refined during training. The efficient one-pixel point rasterization allows us to use arbitrary camera models and display scenes with well over 100M points in real time.

Code on github: https://github.com/darglein/ADOP

More material on Zenodo: https://zenodo.org/record/5602606

More clips in this category "Technische Fakultät"

2024-05-24
Studon
protected  
2024-05-23
IdM-login
protected  
2024-05-23
Free
public  
2024-05-24
Studon
protected