Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network

Abstract:

Large disparity and the non-Lambertian effect challenge light field (LF) reconstruction. Typical approaches either solve the large disparity challenge with depth estimation followed by view synthesis or avoid explicit depth information to enable non-Lambertian rendering, but rarely both. We apply deep learning to the classic LF rendering framework to address both challenges in this paper. First, we analytically demonstrate that the aliasing problem causes the large disparity and non-Lambertian challenges. Classic LF rendering methods use a Fourier reconstruction filter to reduce aliasing, but deep learning pipelines cannot implement it. We introduce an alternative framework for image-domain anti-aliasing reconstruction and analytically demonstrate comparable efficacy. To maximize potential, we embed the anti-aliasing framework into a deep neural network with an integrated architecture and trainable parameters. End-to-end optimization with a unique training set of regular and unstructured LFs trains the network. The proposed deep learning pipeline outperforms other state-of-the-art approaches in solving large disparity and non-Lambertian challenges. The pipeline improves light field view extrapolation as well as LF view interpolation.

Note: Please discuss with our team before submitting this abstract to the college. This Abstract or Synopsis varies based on student project requirements.