Commit Graph
13 Commits
Author SHA1 Message Date
Matthew Hoffman f0aadf9de8 Use ruff as linter and formatter (#2765)
* Switch from black to ruff format

* Sort dependencies with ruff check

* Run ruff format
2024-01-15 13:43:14 -08:00
b17268e83b [WIP] dreamfusion implementation (#1008)
* starting the framework for dreamfusion

* Removed view direction from RGB to calculate albedo

* Started diffusion

* Added dreamfusion to CLI, removed direction embeddings

* Normals in dreamfusion

* Lambertian shading added

* Train normals in Dreamfusion

* Stable diffusion from Stable Dreamfusion

* Fix normal orientation loss

* Configurable light direction sampling

* adding density initalization at origin

* V1 datamanager

* Don't train normals for first 1k iters

* SDS loss added to dreamfusion

* Dreamfusion pipeline

* Background MLP and SphereCollider

* Normals prediction training fixed

* Sigmoid on background, prompt in config

* Shading hyperparameter adjustements

* Shading hyperparameter tuning

* Location based prompting, density initialization fades, LR change

* Cleaned imports, more config options

* Tuning, more config changes

* remove unused sampler, normalize predicted normals from network

* change position encoding to direction encoding for background color mlp

* Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main

* Hard coding less, offloading to config

* Appeasing the linter

* First pass opacity / sparsity losses

* Small patches and configs

* Accidentally pushed logs

* Removing logs for real this time

* Transmittance loss bug fix

* 12 GB Dreamfusion!

* Lint

* Remove unecissary collider changes

* Start SD cleanup

* Cleanup SD code

* adding overhead location prompting

* Use lambertian renderer

* turn grad accum off by default, decrease background mlp size, add overhead text embedding

* Cleanup

* Fix normals vis

* change uniform sampler to prediction network, add option to jit unet for faster sds_loss

* Use shaders

* cleaning up jit code

* add warning if using non jit unet

* add proposal network optimizer to config

* speed up render and cleanup code

* Clean up datamanager

* Add distortion loss

* fix wanb

* adding interpolated location sampling

* Update configs

* interpolated location prompting option, lowering init density, transmittance loss interpolation flipped

* rename alphas mult to alphas loss mult

* Mess around with normals

* add option to use random bg color + bg mlp

* Remove Dreamfusion Trainer (#1423)

* Dreamfusion trainer no longer needed

* Added typing for grad scaler

* SDS return loss

* Fix CFG

* Cleanup

* Update configs

* Fix opacity loss, use unform sampling first round of prop

* add prompt edditing to viewer

* Choose your own diffusion

* remove dreamfusion pipeline file

* merging + remove dreamfusion pipeline file

* Update configs

* add stable diff utils

* cleanup

* cleanup

* Fix test

* positional prompting bugfix

* Warmups

* tune

* lint

* Remove dataparser

* merging + text embedding bugfix

* Set data path to launch viewer for DreamFusion (#1776)

* Set data path to launch viewer for DreamFusion

* Apply formatter

* Fallback to 'base_dir' if data manager has no path

* Make if statement non-inline

* Reformat

---------

Co-authored-by: Matthew Tancik <mattt4001@gmail.com>

* add deepfloyd option first try

* merging

* merging from main

* merging

* removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use

* undo chagnes in data folder

* cleaning code, removing unneccesary changes from main, removed normals prediction network

* remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler

* changing pipeline flag

* move no input data flag from pipeline to datamanager

* undo config and merge in deepfloyd from main

* merging grad scaler pipeline addition

* merging + setting proposal net scheduler

* revert nerfacto config change

* typing + linting

* rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer

* fix test import

* fix test import

* changing datamanager check in trainer, add typing in random cameras datamanager

* add Tensor to randomcamerasdatamanager

* remove old comments

* resolve tyro warning

* update fields to use tcnn

* lint

* add packages for deepfloyd

* add versions

* docs first draft

* rename to generfacto, clean docs

* fix import error

* minor changes

* fix orientation loss bug, retune params

* cleanup

* typing

* trying docs fix

* fixing generative package imports

* fix pyright

* docs fix

* merge w main

* docs minor change

* tuning + bugfixes

* remove torch.backends

---------

Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com>
Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com>
Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
2023-06-17 23:34:04 -07:00
f9a88c5aa2 add deepfloyd model (#1993)
* add deepfloyd model

* black

* lint

* linting

* cast sds gradient to tensor

* change generator type to optional

* lint

* Type fixes

* add typing to sds

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
2023-05-25 00:14:04 -07:00
Matthew Tancik e0e48bcecd Update SD for pytorch 2 (#1943) 2023-05-17 17:47:03 -07:00
Jonáš KulhánekandBrent Yi f481f5ebec Fix some typing problems (#1940)
* Fix some typing problems

* nerfstudio.data typing error fixes

* Add tiny fixes

* Remove ABC from pipeline

* Prevent circular imports

* Tweak ModuleList typing

* TrainingCallbackAttributes cleanup

* Various minor nits

* More nits

* Run `ruff`

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-05-17 17:36:56 -07:00
Brent YiandMatt Tancik d542646c72 Switch to ruff (#1939)
* `isort` -> `ruff`

* `pylint` -> `ruff`

* Remove line

* More minor cleanup

* Better documentation for rules we are enabling

* Fix gitignore, run black

* vscode settings

---------

Co-authored-by: Matt Tancik <mattt4001@gmail.com>
2023-05-17 10:08:14 -07:00
Brent YiandMatt Tancik 6ba2a18b2b torchtyping => jaxtyping (#1838)
* Run `ruff --fix`

* Add jaxtyping migration script

* Run migration scripts + ruff + black

* Merge remote-tracking branch 'origin/main' into brent/jaxtyping

* black

* Fix lint

* fix lint

* Cleanup and more specific types

* Fixed annotation errors

* Fix more errors

* Fix annotation

* Delete _torchtyping_to_jaxtyping.py

---------

Co-authored-by: Matt Tancik <mattt4001@gmail.com>
2023-05-16 15:57:47 -07:00
Ethan Weber 13e0fce122 Update copyright (#1917)
* updated copyright

* including txt file

* updated black

* removing license

* fixing license
2023-05-12 16:16:13 -07:00
Matthew Tancik 4242a3c982 Make rich console global (#1916) 2023-05-12 14:20:40 -07:00
Matthew TancikandDavid McAllister 36514797fd Support multiple versions of stable diffusion (#1461)
Co-authored-by: David McAllister <davidrmc2@gmail.com>
2023-02-20 10:12:41 -08:00
Matthew Tancik 71fd9f7245 Update SDS loss (#1426) 2023-02-14 10:14:48 -08:00
Matthew Tancik e749fbcf78 Trace Stable Diffusion (#1362)
* Trace Stable Diffusion

* pylint

Co-authored by Terrance Wang
2023-02-06 18:33:59 -08:00
11e7fa87e0 Add Stable Diffusion helper code (#1320)
* Add stable diffusion

* Add dependencies fix lint

* Update types

* Update comment

* fix comments

---------

Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
2023-02-01 10:29:08 -08:00