this post was submitted on 26 Oct 2023
12 points (92.9% liked)

Stable Diffusion

4328 readers
46 users here now

Discuss matters related to our favourite AI Art generation technology

Also see

Other communities

founded 1 year ago
MODERATORS
 

Project Page: https://mrtornado24.github.io/DreamCraft3D/

Paper: https://arxiv.org/abs/2310.16818

Git: https://github.com/deepseek-ai/DreamCraft3D

Abstract

We present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation (BSD) to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

top 3 comments
sorted by: hot top controversial new old
[–] [email protected] 5 points 1 year ago

I just need it to output to STL and I can stop trying (and failing) to sculpt 3d prints in blender.

[–] Scew 3 points 1 year ago

Looks solid :D, can't wait for it to be dumbed down to my level! xD

[–] [email protected] 1 points 1 year ago

Here is an alternative Piped link(s):

https://piped.video/0FazXENkQms

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source; check me out at GitHub.