* fixing ns-eval when datamanager is set to disk for splats
* fixing prefetch_factor type issues if user decides to use 0 workers
* removing accidental file
* fixing depth nerfacto config for issue 3592
* ruff linting the imports
* fixing workers creating GPU tensors serialization error
* finished next_eval
* adding flowcharts!
* fixing ruff
* added flowcharts into datamanagers.md
* simplying code and adding GPU serialization fix for both datamanagers
* fixing nits with consistent styling
* last optional nit
* fixing num workers
* initial debugging and testing works
* pwais changes with RayBatchStream to alleviate training
* few bugs to iron out with multiprocessing, specifically pickled collate_fn
* working version of RayBatchStream
* additional docstrings
* cleanup
* much more documentation
* successfully trained AEA-script2_seq2 closed_loop without OOM
* porting over aria dataset-size feature
* added logic to handle eviction of a worker's cached_collated_batch
* antonio's implementation of stream batches
* training on a dataset with 4000 images works!
* some configuration speedups, loops aren't actually needed!
* quick fix adjustment to aria
* removed unnecessary looping
* much faster training when adding i variable to collate every 5 ray bundles
* cleanup unnecssary variables in Dataloader
* further cleanup
* adding caching of compressed images to RAM to reduce disk bottleneck
* added caching to RAM for masks
* found fast way to collate - many tricks applied
* quick update to aria to test on different datasets
* cleaned up the accelerated pil_to_numpy function
* cleaning up PR
* this commit was used to generate the time metrics and profiling metrics
* REAL commit used to run tests
* testing with nerfacto-big
* generated RayBundle collate and converting images from uint8s to float32 on GPU tests
* updating nerfacto to support uint8 easily, will need to figure out a way to contain this within the datamanager API
* datamanager updates, both splat and nerf
* must use writeable arrays because torch requires them
* cleaned up base_dataset, added pickle to utils, more code in full_image, and cleaner desc for base_datamanager
* lots of process on a parallel FullImageDatamanger
* can train big splats with pre-assertion hack or ROI hack and 0 workers
* fixed all undistortion issues with ParallelImageDatamanager
* adding some downsampling and parallel tests with splatfacto!
* deleted commented code in dataloaders.py and added bugfix to shuffling
* testing splatfacto-big
* cleaned up base_pipeline.py
* cleaned up base_pipeline.py ACTUALLY THIS TIME, forgot to save last time
* cleaned up a lot of code
* process_project_aria back to main branch and some cleanup in full_image_datamanager
* clarifying docstrings
* further PR cleanup
* updating models
* further cleanup
* removed caching of images into bytestrings
* adding caching of compressed images to RAM, forgot that hardware matters
* removing oom methods, adding the ability to add a flag to dataloading
* removed CacheDataloader, moved RayBatchStream to dataloaders.py, new vanilla_datamanager rewritten
* fixing base_piplines, deleting a weird datamanager_configs file that was accidently created
* cleaning up next_train
* replaced parallel datamanager with new datamanager
* reverted the original base_datamanager.py, new datamanager replaced parallel_datamanager.py
* modified VanillaConfig, but VanillaDataManager is the same as before
* cleaning up, 2 datamanagers now - original and new parallel one
* able to train with new nerfstudio dataloader now
* side by side datamanagers, moved tons of logic into dataloaders.py and created new files for our parallel datamangers
* added custom ray processing API to support implementations like LERF, cleaned up FullImageDatamanager to original because of new ParallelImageDatamanger
* adding functionality for ns-eval by adding FixedIndicesEvalDataloader to the setup_eval
* adding both ray API and image-view API to datamanagers for custom parallelization
* updating splatfacto config for 4k tests
* updating docstrings to be more descriptive
* new datamanager API breaks when setup_eval() has multiple workers, not sure why but single worker will have to do
* adding custom_view_processor to ImageBatchStream
* reverting full_images_datamanager to main branch
* removing nn.Module inheritance from Datamanager class
* don't need to move datamanger to device anymore since Datamanager is not a subclass of nn.Module
* finished integration test with nerfacto
* simplified config variables, integrated the parallelism/disk-data-loading all into one datamanager
* updated the splatfacto config to be simpler with the dataloading and now uses FullImageDatamanager (which has been changed)
* style checks and some cleanup
* new splatfacto test, cleaning up nerfacto integration test
* removing redundant parallel_full_images_datamaanger, as the OG full_image_datamanager now has full parallelized support
* ruff linting and pyright fixing
* further pyright fixing
* another pyright fixing
* fixing pyright error, camera optimization no longer part of datamanager
* fixing one pyright
* fixing dataloading error when camera is not undistorted with dataloader
* fixing comments and updating style
* undoing a style change i made
* undoing another style change i made by accident
* fixing slow runtime
* fixing a more general camera undistortion bug
* move images to device properly
* minor improvements
* add print statement about >500 images, cleanup method configs
* make method configs consistent across nerfacto models
* adding description comments
* updating description
* resolving some pyright issues with export.py, explained in PR desc
* fixing pyright issues in base_pipeline.py
* ran pyright on exporter and base_pipeline.py without issues
* adding a git ignore to a clearly checked pyright issue
* typo
* fixing most ns-dev-test cases
* cleanup, passing final ns-dev-test
* oops, accidentally pushed the deletion of a docstring, undoing that
* another cleanup
* some fixes to eval pipeline
* lint
* add asserts for spawn
* lint
* cleaning up import statements in parallel_datamanager.py
* adding new developer documentation if users would like to migrate their custom datamanagers to support new features
* removing unnecessary to_device no-op
* further updates to documentation
* lint
* more docs
* docs
* remove comment
* add docs, fix depth dataset with parallel datamanager, fix mask sampling bug
* remove profiling
* more profile removal
* custom_view_processor->custom_image_processor
* doc clarification
* datamanager doc nit
* whitespace
* nits
* remove stuff from __post_init__, tune num workers more, add random offset in raybatchstream
* removing unnecessary assertion, updating docstring because DataManager is no longer an nn.Module
* clarifying configuration with num_images_to_sample_from and num_times_to_repeat_images, cleaning up functions
* adding logic so that nerfacto users can load_from_disk and customize image batch sizes and repeat parameters
* ruff formatting! whoops forgot to format
* fixing logic, now if users set load_from_disk to true, datamanager will use 50 and 10. If users set it and specify their own values, we support that as well
* adding separate datamanager config so that target can be removed in method_configs
---------
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* limited the tyro version up to 0.9.2
* bump tyro>=0.9.8
* Bump typeguard
* Runtime checking with jaxtyping hook instead of typeguard
* Fix test for Python >=3.11, caused by missing default_factory
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
* Update to match viser pr
* Bump viser version
* `done` => `removed` in docs
* Update viewer_elements.py
* Fix
* Point function signature overload
* Update viewer_control.md
* type ignore on callback
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* Support rectangle selection in viewer
* Update doc for click screen_pos
* Raise callback override warning from nerfstudio
* Bump viser version, rect-select only in >0.1.26
* NERFSTUDIO_METHOD_CONFIGS allow method specification as callabale (function or class)
* docs - NERFSTUDIO_METHOD_CONFIGS accepts function or derived class
* Increase max_res of first proposal network
* start on ViewerCamera
* crop controls exposed
* lint
* fix viewer crash when invisible element has its value set
* fov, look_at, and camera type
* stuff
* set update interval correctly in cameracontrols, smooth transitions work now
* demo with depth zooming
* depth min
* WIP adding click message
* put the clock back in viewerwindow
* use setlookat when pos and target are defined at the same time
* remove demo from nerfacto
* Add click commands as rays (origin, direction)
* short doc
* Fix double event for doubleclick
* add ViewerClick, cleanup docs
* add docs
* cleanup
* remove scripts
* remove nerfacto changes
* js compile things
* update viewer version
* fix mutable state thing
* docs improvements
* add kwarg value for enableTransition
* fix
* lint
* path in sync-defs
* Misc doc edits requested in pr
* Add click description
* WIP remove redefinition of banner_height, refer to SCSS vars
* add docstring
* radix
* radix
* speed
---------
Co-authored-by: Chung Min Kim <chungmin99@gmail.com>
* Copy in `viser` communication infrastructure
* Update background image
* Add build directory to fix viser HTTP server
May want to remove these later
* Copy in `viser` communication infrastructure
* Update background image
* Pass dataset vis using viser
* initial impl of ViewerParameter
* idk what i did but brent and matt know
* Suppress eslint typescript warnings
* implement ViewerElement base class, then ViewerButton, ViewerSlider, etc
* client -> server message setup
* multi folder stuff
* Stream controls over viser
* import
* add types to viewer_param.py
* Move another message to viser
* Update folder color
* Use non throttled messages
* multiple folders works
* Cleanup, more porting
* starting impl of control panel in viser
* More porting
* scenebox to viser
* Cleanup
* Fix render updates
* More cleanup, docs, typing
* start control_panel.py
* set_hidden logic but not merged in
* control_panel replicates behavior of existing control panel (visually), except for setting the list of output options. not yet integrated with viewer_utils.py for logic flow. also implemented set_disabled and set_hidden in javscript/ViewerElement side
* Remove some original leva components
* add some train util logic, attempt set_value, doesn't work on javascript side fully yet (value updates but not slider visually
* bye zmq
* missing call to decoder
* 'gui_set' works properly, control panel now replicates behavior of original
* remove print
* Fix status viewer message
* More cleanup
* support set_options in dropdown
* hiding buttons works correctly
* fix set_options, switch from cur_value to get_value()
* Fix slowdown
* output options
* colormap logic converted
* add handle to cb_hook
* fix some lambda bugs
* cleanup
* Save checkpoint logic readded
* Cleanup and minor fixes
* snappier ui
* Refactoring
* Fix state update bugs
* Bug fixes
* Write less to redux
* added comments to reducer code
* Render state machine (#1727)
* state machine works, but train step throttling not implemented
* more step logic, cleanup
* remove random try catcH
* mutex logic for render
* Update lock logic
* types and docs
* added some state machine logic to help rerenders
* merge
---------
Co-authored-by: Justin <justin.g.kerr@gmail.com>
* Refactor and cleanup
* More cleanup and docs
* docstring to gui_utils
* docstrings, rename viewer_param.py to viewer_elements.py
* Fix viewer, remove bridge stuff
* Render on connect
* Fix lint errors
* More eslint fixes
* render_freq logic
* better rerender logic for crop
* More cleanup
* Purge
* Cleanup
* Lint fixes
* Fix cache
* better parsing logic
* Fix infinite test
* More cleanup
* typo
* Update webdocs
* Update docs
* rerender on custom hooks
* add more docs
* Use shared utilities in `viser.infra`
* rework callback to use elements instead of handles
* Fix ESlint errors
* change naming of handle to element, add setter for value
* cleanup
---------
Co-authored-by: Matt Tancik <mattt4001@gmail.com>
Co-authored-by: Justin <justin.g.kerr@gmail.com>
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* launch_eval_blender.sh: fix script
- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated
* launch_train_blender.sh: fix script
- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to https://github.com/nerfstudio-project/nerfstudio/issues/806#issuecomment-1284327844
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line
* launch_eval_blender.sh: fix script
* Update benchmarking.md
* launch_train_blender.sh: add -s to eval command-line
* Update launch_train_blender.sh
* Update benchmarking.md
* alex/record3d_docs_add
* alex/record3d_docs_add
* link from dataparsers to custom_dataset added.
* images are now appearing
Co-authored-by: evonneng <evonne_ng@berkeley.edu>
