Projects

I2DGS: Physically-Based Inverse Rendering with 2D Gaussian Representations

Report: i2DGS — Inverse 2D Gaussian Splatting for OLAT

Mats Grobe, Osama Salem

Technical University of Munich ·

Supervisor Prof. Matthias Nießner

Pipeline diagram: OLAT images of an object are fitted with 2D Gaussians; material and geometry maps (normal, albedo, metallic, roughness) are rasterized into G-buffers, shadow visibility is ray-traced for each light, and both are combined by deferred shading and compared with the captures.
Stage II pipeline. After standard 2DGS training, spherical harmonics are replaced with per-surfel PBR attributes. Material and geometry maps are rasterized into G-buffers, shadow rays are traced to each OLAT light, and deferred shading produces images supervised by the OLAT captures.

TL;DR

Problem
Relightable 3D assets need geometry and spatially varying materials (albedo, roughness, metalness), but radiance-field methods such as NeRF and 3DGS entangle lighting, material and geometry.
Approach
A two-stage pipeline under one-light-at-a-time (OLAT) supervision: 2D Gaussian Splatting first recovers geometry under uniform lighting; then spherical harmonics are replaced by per-surfel PBR attributes, rendered with deferred Disney-style shading, and shadows come from a differentiable, BVH-accelerated ray tracer over the surfels.
Contribution
A fully differentiable 2DGS + PBR inverse-rendering pipeline with ray-traced shadows. On four synthetic OLAT objects it reaches 32.8–35.7 dB relighting PSNR, higher than RNG and GS³ on every object.