* Adding docs for Instruct-GS2GS
Adding documentation for Instruct-GS2GS including its page and adding it to the list of external and 3rd party methods.
* fixing blank formatting dev checks
* updating subtitle
Updating the subtitle to specify gaussian splatting
* Add instructions for using Spectacular AI tools
Adds easy support for new device types: OAK-D, RealSense and
Azure Kinect, as well as an alternative pipeline for iOS.
* Grammar fixes in docs/quickstart/custom_dataset.md (Spectacular AI section)
Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
* Fix section label in the Spectacular AI section of custom_dataset.md
* Update custom_dataset.md: typo fix.
"montions" -> "motions"
---------
Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
* Add support for Project Aria datasets.
1) An export script for processing Project Aria datasets
2) Add support for Fisheye624 cameras
3) Add support for center-crop masking and sampling
Example commands for running nerfstudio on Project Aria data:
python scripts/datasets/process_project_aria.py --vrs_file=<path_to_vrs> --mps_data_dir=<path to MPS directory> --output_dir=<nerfstudio_data_dir>
ns-train nerfacto --data <nerfstudio_data_dir> nerfstudio-data --orientation-method none
* Clean up type errors
* Fix formatting
* Fix linter errors, import, and use absolute file paths
* Add aria subcommand to ns-process-data
* Appease ruff
* Add Aria to docs
* Fix docs, add back import try/except
* nit: path.as_posix() -> str(path)
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* Update URLs for readme and image references
Update the image URLs and documentation links due to change in docs url structure. Files edited include readme, documentation, and legacy viewer files.
* Updating url in custom dataset
Updated the documentation to explicate that zero value in depth images is treated as unknown depth. This behavior is defined in a484d255b4/nerfstudio/model_components/losses.py (L242)
Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
* gh-pages workflow
* update doc.yml
* update
* remove RTD
* change action to main
* test fail on warning
* fail on warnings
* get back .readthedocs.yml, a bit cleanup
* revert SHA of index.md file
* NERFSTUDIO_METHOD_CONFIGS allow method specification as callabale (function or class)
* docs - NERFSTUDIO_METHOD_CONFIGS accepts function or derived class
To prevent abusing of root privilege, please use custom user privilege to start.
If not, sometimes the Docker images may be started with the permissions of root or other users. If you modify the mounted files in the container , it will cause permission errors or other errors.
Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
* nerfbusters dataparser
* nerfbuster dataparser and visualization of eval views
* resolved some errors from pyright
* enable ns-process-data to take two videos as input and use one for eval and the other for train
* resolve some pyright issues
* instead of data being a tuple, I added eval_data as another argument
* implemented train eval split for all cases
* implemented train eval split for all cases
* reverting equi changes
* reverting reality capture changes
* raise valueerror
* revert comment
* added split override code back to dataparser
* reverted one change
* using image border now
* adding docs
---------
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
* Remove system-level CUDA install from Linux install instructions
* Remove system-level CUDA from Windows install instructions
* Un-flip CUDA 11.7 / 11.8
* Moved Volinga as an external method
* Update external_methods.py for black
* Formatted with black
---------
Co-authored-by: jperezh <jperezh@arquimea.com>
* Render VR180 video
Adding the VR180 camera model to render VR180 video and images. I condensed the omnidirectional stereo and VR180 render post processing in the render script. I have updated the documentation for VR180 support.
* Condensing formatting of camera types
Condensing the if statement for camera type intrinsics.
Update to omnidirectional stereo rendering (VR 360) to fix slight positional alignment to accurately composite with Blender objects. Modified documentation for ODS and the Blender add-on to include details on Blender camera settings to composite with the Nerfstudio ODS camera model.
Fixed an error in the documentation where the incorrect path notation was used for Windows systems. The documentation had paths denoted with "./", which is incorrect for Windows. This has been corrected to ".\", the appropriate notation for Windows.
This change ensures that Windows users correctly follow the instructions in the documentation without encountering any path-related issues.
Fix remote viewing setup doc
SSH command should specific localport:remotehost:remoteport rather than localport:localhost:remoteport in -L switch. Changed description for clarity.
* starting the framework for dreamfusion
* Removed view direction from RGB to calculate albedo
* Started diffusion
* Added dreamfusion to CLI, removed direction embeddings
* Normals in dreamfusion
* Lambertian shading added
* Train normals in Dreamfusion
* Stable diffusion from Stable Dreamfusion
* Fix normal orientation loss
* Configurable light direction sampling
* adding density initalization at origin
* V1 datamanager
* Don't train normals for first 1k iters
* SDS loss added to dreamfusion
* Dreamfusion pipeline
* Background MLP and SphereCollider
* Normals prediction training fixed
* Sigmoid on background, prompt in config
* Shading hyperparameter adjustements
* Shading hyperparameter tuning
* Location based prompting, density initialization fades, LR change
* Cleaned imports, more config options
* Tuning, more config changes
* remove unused sampler, normalize predicted normals from network
* change position encoding to direction encoding for background color mlp
* Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main
* Hard coding less, offloading to config
* Appeasing the linter
* First pass opacity / sparsity losses
* Small patches and configs
* Accidentally pushed logs
* Removing logs for real this time
* Transmittance loss bug fix
* 12 GB Dreamfusion!
* Lint
* Remove unecissary collider changes
* Start SD cleanup
* Cleanup SD code
* adding overhead location prompting
* Use lambertian renderer
* turn grad accum off by default, decrease background mlp size, add overhead text embedding
* Cleanup
* Fix normals vis
* change uniform sampler to prediction network, add option to jit unet for faster sds_loss
* Use shaders
* cleaning up jit code
* add warning if using non jit unet
* add proposal network optimizer to config
* speed up render and cleanup code
* Clean up datamanager
* Add distortion loss
* fix wanb
* adding interpolated location sampling
* Update configs
* interpolated location prompting option, lowering init density, transmittance loss interpolation flipped
* rename alphas mult to alphas loss mult
* Mess around with normals
* add option to use random bg color + bg mlp
* Remove Dreamfusion Trainer (#1423)
* Dreamfusion trainer no longer needed
* Added typing for grad scaler
* SDS return loss
* Fix CFG
* Cleanup
* Update configs
* Fix opacity loss, use unform sampling first round of prop
* add prompt edditing to viewer
* Choose your own diffusion
* remove dreamfusion pipeline file
* merging + remove dreamfusion pipeline file
* Update configs
* add stable diff utils
* cleanup
* cleanup
* Fix test
* positional prompting bugfix
* Warmups
* tune
* lint
* Remove dataparser
* merging + text embedding bugfix
* Set data path to launch viewer for DreamFusion (#1776)
* Set data path to launch viewer for DreamFusion
* Apply formatter
* Fallback to 'base_dir' if data manager has no path
* Make if statement non-inline
* Reformat
---------
Co-authored-by: Matthew Tancik <mattt4001@gmail.com>
* add deepfloyd option first try
* merging
* merging from main
* merging
* removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use
* undo chagnes in data folder
* cleaning code, removing unneccesary changes from main, removed normals prediction network
* remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler
* changing pipeline flag
* move no input data flag from pipeline to datamanager
* undo config and merge in deepfloyd from main
* merging grad scaler pipeline addition
* merging + setting proposal net scheduler
* revert nerfacto config change
* typing + linting
* rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer
* fix test import
* fix test import
* changing datamanager check in trainer, add typing in random cameras datamanager
* add Tensor to randomcamerasdatamanager
* remove old comments
* resolve tyro warning
* update fields to use tcnn
* lint
* add packages for deepfloyd
* add versions
* docs first draft
* rename to generfacto, clean docs
* fix import error
* minor changes
* fix orientation loss bug, retune params
* cleanup
* typing
* trying docs fix
* fixing generative package imports
* fix pyright
* docs fix
* merge w main
* docs minor change
* tuning + bugfixes
* remove torch.backends
---------
Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com>
Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com>
Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
* installation.md: Use https instead of ssh for git clone.
Https git clone can be run by anyone directly, without account, whereas ssh git on Github requires having a Github account set up.
* Add .git
---------
Co-authored-by: Matthew Tancik <mattt4001@gmail.com>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* Render Omni-directional stereo (360 3D VR)
Adding the omni-directional stereo camera model to render 3D 360 video and images for VR. This includes the camera models for the left and right eyes and additions to the render script to stack the left and right renders for the final render.
I have also updated the documentation on the custom_dataset page to have information on ODS but let me know if there is a better place to put it.
* Updated ODS camera model and render paths
Updates based on suggested code changes including adding a helper function for ODS and improving path creation.
I also added a part where if a user tries to render a video with an output path that doesn't have an extension it will automatically add .mp4 to the end. Otherwise this usually results in an error when there is no file extension when rendering a video.
* Updated formatting for render and cam path scripts
Updated python formatting for render and camera_paths script
* Adding type checking for ODS function and reformatting camera type code
Added the type checking and doc string for the ODS camera rays function and reformatted the cam_type ray if statements.
