* fixing ns-eval when datamanager is set to disk for splats
* fixing prefetch_factor type issues if user decides to use 0 workers
* removing accidental file
* fixing depth nerfacto config for issue 3592
* ruff linting the imports
* fixing workers creating GPU tensors serialization error
* finished next_eval
* adding flowcharts!
* fixing ruff
* added flowcharts into datamanagers.md
* simplying code and adding GPU serialization fix for both datamanagers
* fixing nits with consistent styling
* last optional nit
* fixing num workers
* initial debugging and testing works
* pwais changes with RayBatchStream to alleviate training
* few bugs to iron out with multiprocessing, specifically pickled collate_fn
* working version of RayBatchStream
* additional docstrings
* cleanup
* much more documentation
* successfully trained AEA-script2_seq2 closed_loop without OOM
* porting over aria dataset-size feature
* added logic to handle eviction of a worker's cached_collated_batch
* antonio's implementation of stream batches
* training on a dataset with 4000 images works!
* some configuration speedups, loops aren't actually needed!
* quick fix adjustment to aria
* removed unnecessary looping
* much faster training when adding i variable to collate every 5 ray bundles
* cleanup unnecssary variables in Dataloader
* further cleanup
* adding caching of compressed images to RAM to reduce disk bottleneck
* added caching to RAM for masks
* found fast way to collate - many tricks applied
* quick update to aria to test on different datasets
* cleaned up the accelerated pil_to_numpy function
* cleaning up PR
* this commit was used to generate the time metrics and profiling metrics
* REAL commit used to run tests
* testing with nerfacto-big
* generated RayBundle collate and converting images from uint8s to float32 on GPU tests
* updating nerfacto to support uint8 easily, will need to figure out a way to contain this within the datamanager API
* datamanager updates, both splat and nerf
* must use writeable arrays because torch requires them
* cleaned up base_dataset, added pickle to utils, more code in full_image, and cleaner desc for base_datamanager
* lots of process on a parallel FullImageDatamanger
* can train big splats with pre-assertion hack or ROI hack and 0 workers
* fixed all undistortion issues with ParallelImageDatamanager
* adding some downsampling and parallel tests with splatfacto!
* deleted commented code in dataloaders.py and added bugfix to shuffling
* testing splatfacto-big
* cleaned up base_pipeline.py
* cleaned up base_pipeline.py ACTUALLY THIS TIME, forgot to save last time
* cleaned up a lot of code
* process_project_aria back to main branch and some cleanup in full_image_datamanager
* clarifying docstrings
* further PR cleanup
* updating models
* further cleanup
* removed caching of images into bytestrings
* adding caching of compressed images to RAM, forgot that hardware matters
* removing oom methods, adding the ability to add a flag to dataloading
* removed CacheDataloader, moved RayBatchStream to dataloaders.py, new vanilla_datamanager rewritten
* fixing base_piplines, deleting a weird datamanager_configs file that was accidently created
* cleaning up next_train
* replaced parallel datamanager with new datamanager
* reverted the original base_datamanager.py, new datamanager replaced parallel_datamanager.py
* modified VanillaConfig, but VanillaDataManager is the same as before
* cleaning up, 2 datamanagers now - original and new parallel one
* able to train with new nerfstudio dataloader now
* side by side datamanagers, moved tons of logic into dataloaders.py and created new files for our parallel datamangers
* added custom ray processing API to support implementations like LERF, cleaned up FullImageDatamanager to original because of new ParallelImageDatamanger
* adding functionality for ns-eval by adding FixedIndicesEvalDataloader to the setup_eval
* adding both ray API and image-view API to datamanagers for custom parallelization
* updating splatfacto config for 4k tests
* updating docstrings to be more descriptive
* new datamanager API breaks when setup_eval() has multiple workers, not sure why but single worker will have to do
* adding custom_view_processor to ImageBatchStream
* reverting full_images_datamanager to main branch
* removing nn.Module inheritance from Datamanager class
* don't need to move datamanger to device anymore since Datamanager is not a subclass of nn.Module
* finished integration test with nerfacto
* simplified config variables, integrated the parallelism/disk-data-loading all into one datamanager
* updated the splatfacto config to be simpler with the dataloading and now uses FullImageDatamanager (which has been changed)
* style checks and some cleanup
* new splatfacto test, cleaning up nerfacto integration test
* removing redundant parallel_full_images_datamaanger, as the OG full_image_datamanager now has full parallelized support
* ruff linting and pyright fixing
* further pyright fixing
* another pyright fixing
* fixing pyright error, camera optimization no longer part of datamanager
* fixing one pyright
* fixing dataloading error when camera is not undistorted with dataloader
* fixing comments and updating style
* undoing a style change i made
* undoing another style change i made by accident
* fixing slow runtime
* fixing a more general camera undistortion bug
* move images to device properly
* minor improvements
* add print statement about >500 images, cleanup method configs
* make method configs consistent across nerfacto models
* adding description comments
* updating description
* resolving some pyright issues with export.py, explained in PR desc
* fixing pyright issues in base_pipeline.py
* ran pyright on exporter and base_pipeline.py without issues
* adding a git ignore to a clearly checked pyright issue
* typo
* fixing most ns-dev-test cases
* cleanup, passing final ns-dev-test
* oops, accidentally pushed the deletion of a docstring, undoing that
* another cleanup
* some fixes to eval pipeline
* lint
* add asserts for spawn
* lint
* cleaning up import statements in parallel_datamanager.py
* adding new developer documentation if users would like to migrate their custom datamanagers to support new features
* removing unnecessary to_device no-op
* further updates to documentation
* lint
* more docs
* docs
* remove comment
* add docs, fix depth dataset with parallel datamanager, fix mask sampling bug
* remove profiling
* more profile removal
* custom_view_processor->custom_image_processor
* doc clarification
* datamanager doc nit
* whitespace
* nits
* remove stuff from __post_init__, tune num workers more, add random offset in raybatchstream
* removing unnecessary assertion, updating docstring because DataManager is no longer an nn.Module
* clarifying configuration with num_images_to_sample_from and num_times_to_repeat_images, cleaning up functions
* adding logic so that nerfacto users can load_from_disk and customize image batch sizes and repeat parameters
* ruff formatting! whoops forgot to format
* fixing logic, now if users set load_from_disk to true, datamanager will use 50 and 10. If users set it and specify their own values, we support that as well
* adding separate datamanager config so that target can be removed in method_configs
---------
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* limited the tyro version up to 0.9.2
* bump tyro>=0.9.8
* Bump typeguard
* Runtime checking with jaxtyping hook instead of typeguard
* Fix test for Python >=3.11, caused by missing default_factory
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
* installation.md: Remove manual gsplat installation for Windows
* installation.md: Reformat and clarify Windows instructions.
* Update installation.md
installation.md: Minor change in wording
---------
Co-authored-by: J.Y. <132313008+jb-ye@users.noreply.github.com>
* removed awscli from deps + added warning during attempted download.
* update docs to flag manual awscli install to users.
* add awscli to dev dependencies
* add proper awscli error handling to ns-download-data
* minor typing fix for ns-download-data when awscli is not installed
* pixi installation optiont
* chore: Update .gitignore to include /third_party and include pixi lock file
* feat: Update download and training commands for dozer data
The commit updates the download and training commands for the dozer data in the `pixi.toml` file. It renames the `download-sample-data` command to `download-dozer-data` and modifies the `train-example-splat` and `train-example-nerf` commands to include the dozer data as a dependency.
This makes it easy to run an example training for both nerfacto and splatfacto
* update to include range between 3.8 -3.10
* Add Pixi installation instructions to the installation.md file
---------
Co-authored-by: maturk <30566358+maturk@users.noreply.github.com>
* bionerf markdown file
* method description in markdown file
* inserting bionerf in index file
* bionerf description
* fixing image paths
* including results on readme
* deleting last line from bionerfs page
* fixing typos in bionerf.md
* bionerf markdown file
* method description in markdown file
* inserting bionerf in index file
* bionerf description
* fixing image paths
* including results on readme
* deleting last line from bionerfs page
* Update to match viser pr
* Bump viser version
* `done` => `removed` in docs
* Update viewer_elements.py
* Fix
* Point function signature overload
* Update viewer_control.md
* type ignore on callback
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* Support rectangle selection in viewer
* Update doc for click screen_pos
* Raise callback override warning from nerfstudio
* Bump viser version, rect-select only in >0.1.26
* fix jittering in markdown in viewer beta
* Revert "fix jittering in markdown in viewer beta"
This reverts commit 70ade42e46.
* print correctly formatted url in banner for viewer beta
* warn when legacy viewerelements are used in new viewer
* Add shim for viewer_elements namespace, fixes https://github.com/ayaanzhaque/instruct-nerf2nerf/pull/88
* Fix typo
* suppress ruff error
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* Setting save_world_frame as False by default
Setting save_world_frame as False by default for the exporter as well as in the viewer, since it is exports the point cloud in a different coordinate system from the Blender add-on and there is not a similar checkbox in the original viewer, so by default the exported point cloud would not be compatible. This flag can still be set true in the cmd command and in the new viewer
Also updating the blender docs to mention support for gs scenes
* Adding splat export panel in viewer
Adding the splat option in the export panel which returns a basic splat export cmd. Also renaming the splat exports as splat.ply instead of point_cloud.ply
* Showing export panels based on method
If gaussian splatting is running, then the crop box, mesh, and point cloud exports are hidden/disabled. Else, all but splat export is visible.
* Fix pyright type check
* Changing viewer model argument for export panels
* Update export_panel.py
* Update export_panel.py
* Update export_panel.py
* Update exporter.py
* Update exporter.py
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
