* GaussianSplats export: allow uint8 in ply
* Ruff formatting
* Add test and narrower typing
* Formatting
* Revert narrow typing, not supported by old python
* Fix typing for old pythons
* Support COLMAP reconstructions with nested image dirs
The code was previously looking for the file.name, which drops all
directory information from the path. Instead, we want to get the file
path relative to the data dir, which will only drop parts of the path
above the data dir, allowing us to recurse into subdirectories within
the images dir
* Adhere to style guidelines
* Add testcase to cover image dirs with subdirs
* ns-process-data updated to output a .ply of sparse points from colmap
* docs for loading 3D points
* format fixing
* ruff formatting
* added read_points3D_text
* formatting
* added 3D points to test
* prompted user bool fix
* small fixes to pass test cases
* fix colors
* fixed coord system
* Tweak frames
* Comment wording
* ruff
* Comment intentional default change for load_3D_points
* Wording
* More wording
* Fix double/float issue
---------
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* add orthophoto CameraType.
* test case of orthophoto camera.
* Fix the rotation errors caused by the usage of left-handed system
* Optimized ortho-cam, it can be used with other camera models now.
* fix TypeError of coords in `test_multi_camera_type()`
---------
Co-authored-by: Alexander Kristoffersen <akristoffersen@berkeley.edu>
* NERFSTUDIO_METHOD_CONFIGS allow method specification as callabale (function or class)
* docs - NERFSTUDIO_METHOD_CONFIGS accepts function or derived class
* nerfbusters dataparser
* nerfbuster dataparser and visualization of eval views
* resolved some errors from pyright
* enable ns-process-data to take two videos as input and use one for eval and the other for train
* resolve some pyright issues
* instead of data being a tuple, I added eval_data as another argument
* implemented train eval split for all cases
* implemented train eval split for all cases
* reverting equi changes
* reverting reality capture changes
* raise valueerror
* revert comment
* added split override code back to dataparser
* reverted one change
* using image border now
* adding docs
---------
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
* Moved Volinga as an external method
* Update external_methods.py for black
* Formatted with black
---------
Co-authored-by: jperezh <jperezh@arquimea.com>
* Implement nerfacto alpha transparency training
* Stop adding all ones alpha channel and clean up blend_background
* Make background_color a required argument in blend_background
* Enable alpha transparent training by default
* Improve background_color typing
* Blend uniform background color for metric computation
* Fix typing error
* Clean up typing
* Fix background_color to outputs assignment
* Model outputs returns unblended RGB (black bg) for random bg
* Update renderers
* Fix comments
* Add support for transparent BG for other methods
* Minor refac
* Fix get_background_color
* Fix typing error for blend_background
* Fix comments
* Export utils support transparency
* Pylint fix
* Fix pcd export masking
* Filter pcd view directions based on opacity
* Clean up get_rgba_image
---------
Co-authored-by: Jonas Kulhanek <jonas.kulhanek@live.com>
* starting the framework for dreamfusion
* Removed view direction from RGB to calculate albedo
* Started diffusion
* Added dreamfusion to CLI, removed direction embeddings
* Normals in dreamfusion
* Lambertian shading added
* Train normals in Dreamfusion
* Stable diffusion from Stable Dreamfusion
* Fix normal orientation loss
* Configurable light direction sampling
* adding density initalization at origin
* V1 datamanager
* Don't train normals for first 1k iters
* SDS loss added to dreamfusion
* Dreamfusion pipeline
* Background MLP and SphereCollider
* Normals prediction training fixed
* Sigmoid on background, prompt in config
* Shading hyperparameter adjustements
* Shading hyperparameter tuning
* Location based prompting, density initialization fades, LR change
* Cleaned imports, more config options
* Tuning, more config changes
* remove unused sampler, normalize predicted normals from network
* change position encoding to direction encoding for background color mlp
* Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main
* Hard coding less, offloading to config
* Appeasing the linter
* First pass opacity / sparsity losses
* Small patches and configs
* Accidentally pushed logs
* Removing logs for real this time
* Transmittance loss bug fix
* 12 GB Dreamfusion!
* Lint
* Remove unecissary collider changes
* Start SD cleanup
* Cleanup SD code
* adding overhead location prompting
* Use lambertian renderer
* turn grad accum off by default, decrease background mlp size, add overhead text embedding
* Cleanup
* Fix normals vis
* change uniform sampler to prediction network, add option to jit unet for faster sds_loss
* Use shaders
* cleaning up jit code
* add warning if using non jit unet
* add proposal network optimizer to config
* speed up render and cleanup code
* Clean up datamanager
* Add distortion loss
* fix wanb
* adding interpolated location sampling
* Update configs
* interpolated location prompting option, lowering init density, transmittance loss interpolation flipped
* rename alphas mult to alphas loss mult
* Mess around with normals
* add option to use random bg color + bg mlp
* Remove Dreamfusion Trainer (#1423)
* Dreamfusion trainer no longer needed
* Added typing for grad scaler
* SDS return loss
* Fix CFG
* Cleanup
* Update configs
* Fix opacity loss, use unform sampling first round of prop
* add prompt edditing to viewer
* Choose your own diffusion
* remove dreamfusion pipeline file
* merging + remove dreamfusion pipeline file
* Update configs
* add stable diff utils
* cleanup
* cleanup
* Fix test
* positional prompting bugfix
* Warmups
* tune
* lint
* Remove dataparser
* merging + text embedding bugfix
* Set data path to launch viewer for DreamFusion (#1776)
* Set data path to launch viewer for DreamFusion
* Apply formatter
* Fallback to 'base_dir' if data manager has no path
* Make if statement non-inline
* Reformat
---------
Co-authored-by: Matthew Tancik <mattt4001@gmail.com>
* add deepfloyd option first try
* merging
* merging from main
* merging
* removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use
* undo chagnes in data folder
* cleaning code, removing unneccesary changes from main, removed normals prediction network
* remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler
* changing pipeline flag
* move no input data flag from pipeline to datamanager
* undo config and merge in deepfloyd from main
* merging grad scaler pipeline addition
* merging + setting proposal net scheduler
* revert nerfacto config change
* typing + linting
* rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer
* fix test import
* fix test import
* changing datamanager check in trainer, add typing in random cameras datamanager
* add Tensor to randomcamerasdatamanager
* remove old comments
* resolve tyro warning
* update fields to use tcnn
* lint
* add packages for deepfloyd
* add versions
* docs first draft
* rename to generfacto, clean docs
* fix import error
* minor changes
* fix orientation loss bug, retune params
* cleanup
* typing
* trying docs fix
* fixing generative package imports
* fix pyright
* docs fix
* merge w main
* docs minor change
* tuning + bugfixes
* remove torch.backends
---------
Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu>
Co-authored-by: David McAllister <davidrmc2@gmail.com>
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com>
Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com>
Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
* Fix model loading collate_fn non-static
* Fix serialization
* Add tests to reproduce the error
* Comment the change to let the tests fail
* Finish changes
* started a package to easily swap out tcnn for pytorch
* trains with pytorch and not tcnn
* cleaned up imports
* making it easy to install on mac
* removed/reverted files
* merged main
* reverted changes
* updating config
* starting to add nerfacto to tests
* fixxing default value
* fixed default implementation
* defaults to torch if tcnn and not installed
* update log
* add comment
* resolved pr comments
* fix
* update default
---------
Co-authored-by: Ethan Weber <ethanweber@eecs-128-32-175-250.dhcp.berkeley.edu>
* knocking out a couple
* Fix more errors
* Add pyright
* Remove some unnecessary cv2 type ignores, warning on missing imports
(for missing tinycudann)
* Minor annotation fixes
* Update docs
* Drop `None` from temporal distortion output
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* `isort` -> `ruff`
* `pylint` -> `ruff`
* Remove line
* More minor cleanup
* Better documentation for rules we are enabling
* Fix gitignore, run black
* vscode settings
---------
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
* implementing l1 and tv reg in the original tensorf paper
* added documentation for tv loss
* added test for tv loss
* fix formating
* fix formatting
---------
Co-authored-by: Yan Xiao <yax010@ucsd.edu>
* Refactor of image and video dataset conversion
The classes to convert images and video to a NerfStudio
dataset are moved to the main repo and refactor:
* Code duplications are removed
* Import of the class is now done from nerfstudio package instead of
scripts
* Doc formatting nits
* Doc formatting
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* profiling
* set pose_optimizer to None if it's mode is "off"
* radial_and_tangential_undistort in nerfacc
* fix camera distortions
* update to nerfacc 0.5.1. Upgrade instant-ngp model.
* revert nerfacc to 0.5.1
* format
* allows to fall back to the pytorch undistortion
* format
* fix linting
* fix test
* fix test
* fix nerfplayer-ngp
* enable_collider=False for NGP
* nerfplayer verified.
* zero cone_angle for ngp bounded
* fix format
* fix pylint
* Copy in `viser` communication infrastructure
* Update background image
* Add build directory to fix viser HTTP server
May want to remove these later
* Copy in `viser` communication infrastructure
* Update background image
* Pass dataset vis using viser
* initial impl of ViewerParameter
* idk what i did but brent and matt know
* Suppress eslint typescript warnings
* implement ViewerElement base class, then ViewerButton, ViewerSlider, etc
* client -> server message setup
* multi folder stuff
* Stream controls over viser
* import
* add types to viewer_param.py
* Move another message to viser
* Update folder color
* Use non throttled messages
* multiple folders works
* Cleanup, more porting
* starting impl of control panel in viser
* More porting
* scenebox to viser
* Cleanup
* Fix render updates
* More cleanup, docs, typing
* start control_panel.py
* set_hidden logic but not merged in
* control_panel replicates behavior of existing control panel (visually), except for setting the list of output options. not yet integrated with viewer_utils.py for logic flow. also implemented set_disabled and set_hidden in javscript/ViewerElement side
* Remove some original leva components
* add some train util logic, attempt set_value, doesn't work on javascript side fully yet (value updates but not slider visually
* bye zmq
* missing call to decoder
* 'gui_set' works properly, control panel now replicates behavior of original
* remove print
* Fix status viewer message
* More cleanup
* support set_options in dropdown
* hiding buttons works correctly
* fix set_options, switch from cur_value to get_value()
* Fix slowdown
* output options
* colormap logic converted
* add handle to cb_hook
* fix some lambda bugs
* cleanup
* Save checkpoint logic readded
* Cleanup and minor fixes
* snappier ui
* Refactoring
* Fix state update bugs
* Bug fixes
* Write less to redux
* added comments to reducer code
* Render state machine (#1727)
* state machine works, but train step throttling not implemented
* more step logic, cleanup
* remove random try catcH
* mutex logic for render
* Update lock logic
* types and docs
* added some state machine logic to help rerenders
* merge
---------
Co-authored-by: Justin <justin.g.kerr@gmail.com>
* Refactor and cleanup
* More cleanup and docs
* docstring to gui_utils
* docstrings, rename viewer_param.py to viewer_elements.py
* Fix viewer, remove bridge stuff
* Render on connect
* Fix lint errors
* More eslint fixes
* render_freq logic
* better rerender logic for crop
* More cleanup
* Purge
* Cleanup
* Lint fixes
* Fix cache
* better parsing logic
* Fix infinite test
* More cleanup
* typo
* Update webdocs
* Update docs
* rerender on custom hooks
* add more docs
* Use shared utilities in `viser.infra`
* rework callback to use elements instead of handles
* Fix ESlint errors
* change naming of handle to element, add setter for value
* cleanup
---------
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: Justin <justin.g.kerr@gmail.com>
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* add neus_facto method
* fix missing pixsfm from video process
* fix dev errors and small typos
---------
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
* Increase max_res of first proposal network
* nerfacto-big params
* skip nerfacto_big in test
---------
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
* fix checkpoint generated by ddp training cannot be loaded
* fix typo
* reformatted nerfstudio/pipelines/base_pipeline.py
* rewritten as for loop
* add test case
* reformatted
* fix pylint test failed
* add initial method configs and data manager
* add neus scheduler
* add initial surface model and neus
* add Neus ray sampler
* add sdf field
* move scale/shift calc to math utils
* add debug and fix sdf_dataset typo
* update vscode neus debug
* fix dataset and get initial working version
* add magic number from original repo
* black fixes
* clean up get weights/transmittance from alphas
* clean up typos in sdf datamanager
* type annotations on neus scheduler and remove unused params
* clean up and annotate sdf field along w/ other fixes
* more clean up
* remove unnessary directions norm
* resolve comments
* update neus scheduler to follow refactor
* fix linter errors
* fix some typos
* remove get_weights_from_alpha
* more comment fixes
* more linter error fixes
* clean up and add comments
* more little fixes for types/comments
* more comments
* rename NeuS Scheduler to DelayedScheduler
* rename DelayedScheduler to CosineDecayScheduler
