Commit Graph
1715 Commits
Author SHA1 Message Date
Weihang Li c7def8ce8c Fix image saving format issue in render (#2172)
* fix scannet dataparser pose issue

* camera pose corrected

* fix image format saving issue
2023-07-04 14:07:04 -07:00
KaerMorh 18ab50dc6b Correct path notation for Windows in documentation (#2171)
Fixed an error in the documentation where the incorrect path notation was used for Windows systems. The documentation had paths denoted with "./", which is incorrect for Windows. This has been corrected to ".\", the appropriate notation for Windows.

This change ensures that Windows users correctly follow the instructions in the documentation without encountering any path-related issues.
2023-07-04 08:14:15 -07:00
Daniel CrispellandBrent Yi 8ed2a2d667 Make all SDFFieldConfig params configurable via constructor (#2167)
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-07-03 21:00:38 +00:00
Matthew Tancik 061d691d43 Default to tcnn (#2159) 2023-07-02 21:36:28 -07:00
Brent Yi d6bb8bd8e5 Fix use_appearance_embedding for instant-ngp (#1999)
* Attempt fixes for instant-ngp

* Run `ns-dev-test`

* Remove NGP field implementation

* Test nerfacto field instead of ngp field

* Black
2023-07-01 18:54:27 +00:00
Ilya Basharov 5c4b0a1dd6 add build-arg cuda version to dockerfile (#2139) 2023-07-01 17:06:05 +00:00
maturk 37e763f2de optional pose noise in datamanager 2023-06-30 14:09:57 +03:00
maturk 4c3db790a4 barf 2023-06-30 14:08:05 +03:00
Milan van Wouden f4be2b729a Fix typo in nerfacto.py (#2149) 2023-06-30 08:44:53 +00:00
hturkiandMatthew Tancik 0719026c15 Add option to render evaluated images (#2135)
* Add option to render evaluated images

* add args

* fix formatting

* attempt to fix pyright

* attempt 2

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-06-29 23:26:56 +00:00
Jonáš KulhánekandMatthew Tancik b3df46314c Remove breakpoint from colmap_dataparser (#2141)
Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-06-29 17:03:29 +00:00
Konstantin Kozlovtsev 000272f895 Fix unnecessary encodings calculation in tensorf field (#2145)
Fixed unnecessary encodings calculation in tensorf field
2023-06-29 09:54:59 -07:00
machenmusikandMatias Turkulainen d78af5b569 allow ns-process-data to use CR2 raw images (#2119)
* allow ns-process-data to use CR2 raw images

* Update process_data_utils.py

* black wants an extra newline

---------

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
2023-06-27 16:02:58 -04:00
Ethan WeberandMatias Turkulainen d9ee5752d2 Fix to fall back to torch if tcnn not found (#2124)
fix for falling back to torch if tcnn not found

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
2023-06-27 09:33:28 -07:00
maturk d1350e1ecf max num threads as config parameter 2023-06-27 14:23:03 +03:00
maturk 5f85ef181d data queue as config parameter 2023-06-27 13:55:40 +03:00
Matias TurkulainenandPaul Wais 7121bc8c47 docstring
Co-authored-by: Paul Wais <paulwais@gmail.com>
2023-06-27 10:07:19 +03:00
maturk 10fc5f3657 explicitly catch full queue, notify of other exceptions 2023-06-26 15:21:13 +03:00
Matthew Tancik 6486c00d4d Update generfacto.md (#2132) 2023-06-25 23:48:33 -07:00
Jonáš Kulhánek aa6c052660 Jkulhanek/colmap dataparser (#2115)
* Add COLMAP dataparser implementation

* Fix downscale_images input

* Add help to COLMAP dataparser
2023-06-25 17:08:17 +00:00
Jonáš Kulhánek ac6a83cc53 Fix pickling of VanillaDataManager (#2129)
* Fix pickling of VanillaDataManager

* Fix vanilladatamanager init
2023-06-25 18:46:07 +02:00
Matthew Tancik 004d8ca9d2 Update nerfacto.md (#2126) 2023-06-23 17:56:11 -07:00
olegpokhilchenko ef1350bd36 Fix get_num_frames_in_video (#2090)
* Fix get_num_frames_in_video

* Make sure that there is a number

* Fix output[0]
2023-06-23 08:38:16 -07:00
Terrance WangandMatthew Tancik c023f82b11 Add examps to generfacto docs (#2120)
* add vids to generfacto docs

* update generfacto docs

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-06-22 21:24:29 +00:00
Matthew Tancik 7349f2320b v0.3.2 (#2118) v0.3.2 2023-06-22 09:25:36 -07:00
jlightfoot f22e9f4db9 Fix quickstart doc for remote host viewing setup (#2101)
Fix remote viewing setup doc

SSH command should specific localport:remotehost:remoteport rather than localport:localhost:remoteport in -L switch.  Changed description for clarity.
2023-06-22 16:22:45 +00:00
Jonáš Kulhánek 3214fcfeb5 Fix spherical harmonics #2081 (#2112) 2023-06-22 02:40:22 +00:00
Jonáš Kulhánek c68d622bbc Allow newer pycolmap for newer python (#2113) 2023-06-21 19:23:34 -07:00
Justin 3f73e78bfd oops 2023-06-21 22:26:27 +00:00
Justin 1d8cef26f7 clean 2023-06-21 22:25:01 +00:00
Justin a73d892f21 params 2023-06-21 21:02:07 +00:00
Justin 09e9a71d3d minor speedup 2023-06-21 20:46:20 +00:00
Justin 75d1013d60 merge 2023-06-21 18:51:02 +00:00
Justin e26bd4d525 initial impl of parallel data manager 2023-06-21 18:50:22 +00:00
b747db6877 Fixing the image_dir if using skip_image_processing (#2111)
Co-authored-by: khoa <tuankhoanguyen94@gmail.com>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-06-21 12:24:55 -04:00
machenmusik ccb8a25d4a Add option to process masks on GPU for speed at the expense of memory (#2110)
--pipeline.datamanager.masks-on-gpu True
2023-06-21 09:16:14 -07:00
maturk f6b6ed8ca8 ensure that non_trainable_indices are on the correct device during training 2023-06-21 17:47:52 +03:00
Shiva Gantha 8cdec3e1d2 Update nerfacto.md (#2107) 2023-06-20 19:39:11 -07:00
Justin 807837a1dc merge main 2023-06-20 06:17:15 +00:00
Justin caf50a751a Merge branch 'main' of https://github.com/nerfstudio-project/nerfstudio 2023-06-20 05:46:06 +00:00
Justin 767dc100d1 param 2023-06-20 05:45:58 +00:00
Matias TurkulainenandMatthew Tancik e0e66ce9fb Add non-trainable camera indices to CameraOptimizer (#2080)
* add non-trainable indices to camera optimizer

* improve comment

* create on device

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-06-18 12:55:13 -04:00
ksnzhandZhang Jian c55f5d36b9 add gradient accumulation support (#2049)
* add gradient accumulation support

* fix 2 blank lines

* fix possibly unbound variable

* update gradient accumulation step with for loop

* add seert and ignore pyright check

---------

Co-authored-by: Zhang Jian <zhangjian49@lenovo.com>
2023-06-18 07:17:35 -07:00
Justin b9feec2a0e param 2023-06-18 07:05:13 +00:00
b17268e83b [WIP] dreamfusion implementation (#1008)
* starting the framework for dreamfusion

* Removed view direction from RGB to calculate albedo

* Started diffusion

* Added dreamfusion to CLI, removed direction embeddings

* Normals in dreamfusion

* Lambertian shading added

* Train normals in Dreamfusion

* Stable diffusion from Stable Dreamfusion

* Fix normal orientation loss

* Configurable light direction sampling

* adding density initalization at origin

* V1 datamanager

* Don't train normals for first 1k iters

* SDS loss added to dreamfusion

* Dreamfusion pipeline

* Background MLP and SphereCollider

* Normals prediction training fixed

* Sigmoid on background, prompt in config

* Shading hyperparameter adjustements

* Shading hyperparameter tuning

* Location based prompting, density initialization fades, LR change

* Cleaned imports, more config options

* Tuning, more config changes

* remove unused sampler, normalize predicted normals from network

* change position encoding to direction encoding for background color mlp

* Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main

* Hard coding less, offloading to config

* Appeasing the linter

* First pass opacity / sparsity losses

* Small patches and configs

* Accidentally pushed logs

* Removing logs for real this time

* Transmittance loss bug fix

* 12 GB Dreamfusion!

* Lint

* Remove unecissary collider changes

* Start SD cleanup

* Cleanup SD code

* adding overhead location prompting

* Use lambertian renderer

* turn grad accum off by default, decrease background mlp size, add overhead text embedding

* Cleanup

* Fix normals vis

* change uniform sampler to prediction network, add option to jit unet for faster sds_loss

* Use shaders

* cleaning up jit code

* add warning if using non jit unet

* add proposal network optimizer to config

* speed up render and cleanup code

* Clean up datamanager

* Add distortion loss

* fix wanb

* adding interpolated location sampling

* Update configs

* interpolated location prompting option, lowering init density, transmittance loss interpolation flipped

* rename alphas mult to alphas loss mult

* Mess around with normals

* add option to use random bg color + bg mlp

* Remove Dreamfusion Trainer (#1423)

* Dreamfusion trainer no longer needed

* Added typing for grad scaler

* SDS return loss

* Fix CFG

* Cleanup

* Update configs

* Fix opacity loss, use unform sampling first round of prop

* add prompt edditing to viewer

* Choose your own diffusion

* remove dreamfusion pipeline file

* merging + remove dreamfusion pipeline file

* Update configs

* add stable diff utils

* cleanup

* cleanup

* Fix test

* positional prompting bugfix

* Warmups

* tune

* lint

* Remove dataparser

* merging + text embedding bugfix

* Set data path to launch viewer for DreamFusion (#1776)

* Set data path to launch viewer for DreamFusion

* Apply formatter

* Fallback to 'base_dir' if data manager has no path

* Make if statement non-inline

* Reformat

---------

Co-authored-by: Matthew Tancik <mattt4001@gmail.com>

* add deepfloyd option first try

* merging

* merging from main

* merging

* removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use

* undo chagnes in data folder

* cleaning code, removing unneccesary changes from main, removed normals prediction network

* remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler

* changing pipeline flag

* move no input data flag from pipeline to datamanager

* undo config and merge in deepfloyd from main

* merging grad scaler pipeline addition

* merging + setting proposal net scheduler

* revert nerfacto config change

* typing + linting

* rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer

* fix test import

* fix test import

* changing datamanager check in trainer, add typing in random cameras datamanager

* add Tensor to randomcamerasdatamanager

* remove old comments

* resolve tyro warning

* update fields to use tcnn

* lint

* add packages for deepfloyd

* add versions

* docs first draft

* rename to generfacto, clean docs

* fix import error

* minor changes

* fix orientation loss bug, retune params

* cleanup

* typing

* trying docs fix

* fixing generative package imports

* fix pyright

* docs fix

* merge w main

* docs minor change

* tuning + bugfixes

* remove torch.backends

---------

Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com>
Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com>
Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
2023-06-17 23:34:04 -07:00
Justin 75153f6679 update viewer beta vis 2023-06-18 06:32:40 +00:00
Justin 2fb9cd7cb9 Merge branch 'justin/camera_clicking_beta' into justin/camera_opt_refactor 2023-06-18 06:09:52 +00:00
Justin be47ffca81 add l2 penalties on optimizers 2023-06-18 06:09:46 +00:00
Justin ae5ef15bed rip out camera optimization from datamanager and move to nerfacto.py 2023-06-18 05:43:31 +00:00
Justin 1182b18b48 Merge branch 'main' of https://github.com/nerfstudio-project/nerfstudio 2023-06-18 05:06:40 +00:00