Get depth and new views from one image by coding for shape, appearance, pose with im2nerf

Get depth and new views from one image by coding for shape, appearance, pose with im2nerf

im2nerf: Image to Neural Radiance Field in the Wild

arXiv paper abstract https://arxiv.org/abs/2209.04061v1

arXiv PDF paper https://arxiv.org/pdf/2209.04061v1.pdf

... propose im2nerf, a learning framework that predicts a continuous neural object representation given a single input image in the wild, supervised by only segmentation output from off-the-shelf recognition methods.

The standard approach to constructing neural radiance fields takes advantage of multi-view consistency and requires many calibrated views of a scene, a requirement that cannot be satisfied when learning on large-scale image data in the wild.

... introducing a model that encodes the input image into a disentangled object representation that contains a code for object shape, a code for object appearance, and an estimated camera pose from which the object image is captured.

... model conditions a NeRF on the predicted object representation and uses volume rendering to generate images from novel views.

... in addition to using a reconstruction loss on the synthesized input view, ... use an auxiliary adversarial loss on the novel rendered views.

... im2nerf achieves the state-of-the-art performance for novel view synthesis from a single-view unposed image in the wild.

Please like and share this post if you enjoyed it using the buttons at the bottom! Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact Web site with my other posts by category https://morrislee1234.wixsite.com/website LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b #ComputerVision #3D #AINewsClips #AI #ML #ArtificialIntelligence #MachineLearning