* Changed torch version to <=2.2, as <=2.1 will uninstall torch 2.1.2+cu118 and leave users without cuda in torch
* changed <=2.2 to <2.2 to prevent the issue at to prevent the type error referenced here https://github.com/nerfstudio-project/nerfstudio/pull/2857
* Changed torch version to <=2.2, as <=2.1 will uninstall torch 2.1.2+cu118 and leave users without cuda in torch
* Add ability to download EyefulTower dataset
* wip before I copy linning's stuff in
* Generate per-resolution cameras.xml
* Generate transforms.json at download
* Fix a couple of quotes
* Use official EyefulTower splits for train and val
* Disable projectaria-tools on windows
* Fix extra imports
* Add a new nerfacto method tund for EyefulTower
* Split eyefultower download into a separate file
* Add some fisheye support for eyeful data
* Reformatted imports to not be dumb
* Apparently this file was missed when formatting originally
* Added 1k resolution scenes
* revert method_configs.py to original values
* Also add 1k exrs
* Address feedback
* Revert changes to pyproject.toml, to be added in a later PR
* Oops, probably shouldn't have gotten rid of awscli ...
* changed glob variable name
* Update tensor_dataclass.py
---------
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* undistortion should not occur if pinhole (smoothes the images), and accordingly fix rounding of downscaled image size
* rounding downscaling, more consistent with other resizing methods in public datasets
* reverting to cpu image caching
* re-allow camera unimplemented
* compatibility with mask/depth
* reverted scale factor rounding, corrected mask undistort condition
* moved distortion_params update inside PERSPECTIVE type if-clause
* bumping opencv to 4.8, to deal with undistortion bug with distortion parameters of zero
---------
Co-authored-by: liorf <liorf@amazon.com>
Co-authored-by: alshan <alshan@amazon.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
* Add support for Project Aria datasets.
1) An export script for processing Project Aria datasets
2) Add support for Fisheye624 cameras
3) Add support for center-crop masking and sampling
Example commands for running nerfstudio on Project Aria data:
python scripts/datasets/process_project_aria.py --vrs_file=<path_to_vrs> --mps_data_dir=<path to MPS directory> --output_dir=<nerfstudio_data_dir>
ns-train nerfacto --data <nerfstudio_data_dir> nerfstudio-data --orientation-method none
* Clean up type errors
* Fix formatting
* Fix linter errors, import, and use absolute file paths
* Add aria subcommand to ns-process-data
* Appease ruff
* Add Aria to docs
* Fix docs, add back import try/except
* nit: path.as_posix() -> str(path)
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* Fix large dataset (> 512 images) training in viewer_beta (#2586)
* fix index bug in update_camera_poses of viewer
* Set Ruff line lenth
---------
Co-authored-by: Gina Wu <42229107+ginazhouhuiwu@users.noreply.github.com>
* Add share link support for beta viewer
* More consistent CLI experience
* fix viewer jitter by adding atomic update on camera pose
* change to make_share_url
* fix
* remove incorrect assert
---------
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
* added pytorch omnicv implementation in order to not use GPL equi2pers
* renamed rotation matrix function to decrease confusion
* fixed docstrings
* fixed linting
* removed pyequilib from dependencies
* allow ns-process-data to use CR2 raw images
* Update process_data_utils.py
* black wants an extra newline
---------
Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
* starting the framework for dreamfusion
* Removed view direction from RGB to calculate albedo
* Started diffusion
* Added dreamfusion to CLI, removed direction embeddings
* Normals in dreamfusion
* Lambertian shading added
* Train normals in Dreamfusion
* Stable diffusion from Stable Dreamfusion
* Fix normal orientation loss
* Configurable light direction sampling
* adding density initalization at origin
* V1 datamanager
* Don't train normals for first 1k iters
* SDS loss added to dreamfusion
* Dreamfusion pipeline
* Background MLP and SphereCollider
* Normals prediction training fixed
* Sigmoid on background, prompt in config
* Shading hyperparameter adjustements
* Shading hyperparameter tuning
* Location based prompting, density initialization fades, LR change
* Cleaned imports, more config options
* Tuning, more config changes
* remove unused sampler, normalize predicted normals from network
* change position encoding to direction encoding for background color mlp
* Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main
* Hard coding less, offloading to config
* Appeasing the linter
* First pass opacity / sparsity losses
* Small patches and configs
* Accidentally pushed logs
* Removing logs for real this time
* Transmittance loss bug fix
* 12 GB Dreamfusion!
* Lint
* Remove unecissary collider changes
* Start SD cleanup
* Cleanup SD code
* adding overhead location prompting
* Use lambertian renderer
* turn grad accum off by default, decrease background mlp size, add overhead text embedding
* Cleanup
* Fix normals vis
* change uniform sampler to prediction network, add option to jit unet for faster sds_loss
* Use shaders
* cleaning up jit code
* add warning if using non jit unet
* add proposal network optimizer to config
* speed up render and cleanup code
* Clean up datamanager
* Add distortion loss
* fix wanb
* adding interpolated location sampling
* Update configs
* interpolated location prompting option, lowering init density, transmittance loss interpolation flipped
* rename alphas mult to alphas loss mult
* Mess around with normals
* add option to use random bg color + bg mlp
* Remove Dreamfusion Trainer (#1423)
* Dreamfusion trainer no longer needed
* Added typing for grad scaler
* SDS return loss
* Fix CFG
* Cleanup
* Update configs
* Fix opacity loss, use unform sampling first round of prop
* add prompt edditing to viewer
* Choose your own diffusion
* remove dreamfusion pipeline file
* merging + remove dreamfusion pipeline file
* Update configs
* add stable diff utils
* cleanup
* cleanup
* Fix test
* positional prompting bugfix
* Warmups
* tune
* lint
* Remove dataparser
* merging + text embedding bugfix
* Set data path to launch viewer for DreamFusion (#1776)
* Set data path to launch viewer for DreamFusion
* Apply formatter
* Fallback to 'base_dir' if data manager has no path
* Make if statement non-inline
* Reformat
---------
Co-authored-by: Matthew Tancik <mattt4001@gmail.com>
* add deepfloyd option first try
* merging
* merging from main
* merging
* removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use
* undo chagnes in data folder
* cleaning code, removing unneccesary changes from main, removed normals prediction network
* remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler
* changing pipeline flag
* move no input data flag from pipeline to datamanager
* undo config and merge in deepfloyd from main
* merging grad scaler pipeline addition
* merging + setting proposal net scheduler
* revert nerfacto config change
* typing + linting
* rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer
* fix test import
* fix test import
* changing datamanager check in trainer, add typing in random cameras datamanager
* add Tensor to randomcamerasdatamanager
* remove old comments
* resolve tyro warning
* update fields to use tcnn
* lint
* add packages for deepfloyd
* add versions
* docs first draft
* rename to generfacto, clean docs
* fix import error
* minor changes
* fix orientation loss bug, retune params
* cleanup
* typing
* trying docs fix
* fixing generative package imports
* fix pyright
* docs fix
* merge w main
* docs minor change
* tuning + bugfixes
* remove torch.backends
---------
Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com>
Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com>
Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
J.Y
Matthew Tancik
Brent Yi
Simon Bethke
Justin Kerr
Chung Min Kim
Too Yuen MinandBrent Yi
Matthew Hoffman
Ruilong Li(李瑞龙)
Chris SweeneyandBrent Yi
Sid Mehta
Rohan Mathur
Jonáš Kulhánek
