Projects

Seminar: GaussianDreamer — Fast Text-to-3D with Gaussian Splatting

Osama Salem

Seminar on Neural Radiance Fields and Gaussian Splatting, Technical University of Munich ·

Supervisors Jiapeng Tang, Can Gümeli, Prof. Matthias Nießner

A text prompt 'A spiderman bust…' is turned into a 3D Gaussian bust that sharpens from initialization through 2, 7 and 14 minutes, followed by rendered views.
GaussianDreamer overview (figure from the seminar report, after Yi et al.): from a text prompt, the 3D Gaussians are initialized and refined within minutes.

TL;DR

Problem
Text-to-3D methods driven by 2D diffusion models produce detailed results but are slow and often 3D-inconsistent, while 3D diffusion models are fast but limited in quality.
Approach
The reviewed method initializes 3D Gaussians from a 3D diffusion prior (Shap-E, or MDM for human motion), densifies them with noisy point growing and colour perturbation, and refines them with score distillation from a 2D diffusion model — about 15 minutes on one GPU, with real-time rendering.
This seminar
A report and talk that place GaussianDreamer in the context of diffusion models, NeRF/DreamFusion and 3D Gaussian Splatting, and discuss its strengths and limitations. The method itself is the work of Yi et al. (paper).