* fixing ns-eval when datamanager is set to disk for splats
* fixing prefetch_factor type issues if user decides to use 0 workers
* removing accidental file
* fixing depth nerfacto config for issue 3592
* ruff linting the imports
* fixing workers creating GPU tensors serialization error
* finished next_eval
* adding flowcharts!
* fixing ruff
* added flowcharts into datamanagers.md
* simplying code and adding GPU serialization fix for both datamanagers
* fixing nits with consistent styling
* last optional nit
* fixing num workers
* initial debugging and testing works
* pwais changes with RayBatchStream to alleviate training
* few bugs to iron out with multiprocessing, specifically pickled collate_fn
* working version of RayBatchStream
* additional docstrings
* cleanup
* much more documentation
* successfully trained AEA-script2_seq2 closed_loop without OOM
* porting over aria dataset-size feature
* added logic to handle eviction of a worker's cached_collated_batch
* antonio's implementation of stream batches
* training on a dataset with 4000 images works!
* some configuration speedups, loops aren't actually needed!
* quick fix adjustment to aria
* removed unnecessary looping
* much faster training when adding i variable to collate every 5 ray bundles
* cleanup unnecssary variables in Dataloader
* further cleanup
* adding caching of compressed images to RAM to reduce disk bottleneck
* added caching to RAM for masks
* found fast way to collate - many tricks applied
* quick update to aria to test on different datasets
* cleaned up the accelerated pil_to_numpy function
* cleaning up PR
* this commit was used to generate the time metrics and profiling metrics
* REAL commit used to run tests
* testing with nerfacto-big
* generated RayBundle collate and converting images from uint8s to float32 on GPU tests
* updating nerfacto to support uint8 easily, will need to figure out a way to contain this within the datamanager API
* datamanager updates, both splat and nerf
* must use writeable arrays because torch requires them
* cleaned up base_dataset, added pickle to utils, more code in full_image, and cleaner desc for base_datamanager
* lots of process on a parallel FullImageDatamanger
* can train big splats with pre-assertion hack or ROI hack and 0 workers
* fixed all undistortion issues with ParallelImageDatamanager
* adding some downsampling and parallel tests with splatfacto!
* deleted commented code in dataloaders.py and added bugfix to shuffling
* testing splatfacto-big
* cleaned up base_pipeline.py
* cleaned up base_pipeline.py ACTUALLY THIS TIME, forgot to save last time
* cleaned up a lot of code
* process_project_aria back to main branch and some cleanup in full_image_datamanager
* clarifying docstrings
* further PR cleanup
* updating models
* further cleanup
* removed caching of images into bytestrings
* adding caching of compressed images to RAM, forgot that hardware matters
* removing oom methods, adding the ability to add a flag to dataloading
* removed CacheDataloader, moved RayBatchStream to dataloaders.py, new vanilla_datamanager rewritten
* fixing base_piplines, deleting a weird datamanager_configs file that was accidently created
* cleaning up next_train
* replaced parallel datamanager with new datamanager
* reverted the original base_datamanager.py, new datamanager replaced parallel_datamanager.py
* modified VanillaConfig, but VanillaDataManager is the same as before
* cleaning up, 2 datamanagers now - original and new parallel one
* able to train with new nerfstudio dataloader now
* side by side datamanagers, moved tons of logic into dataloaders.py and created new files for our parallel datamangers
* added custom ray processing API to support implementations like LERF, cleaned up FullImageDatamanager to original because of new ParallelImageDatamanger
* adding functionality for ns-eval by adding FixedIndicesEvalDataloader to the setup_eval
* adding both ray API and image-view API to datamanagers for custom parallelization
* updating splatfacto config for 4k tests
* updating docstrings to be more descriptive
* new datamanager API breaks when setup_eval() has multiple workers, not sure why but single worker will have to do
* adding custom_view_processor to ImageBatchStream
* reverting full_images_datamanager to main branch
* removing nn.Module inheritance from Datamanager class
* don't need to move datamanger to device anymore since Datamanager is not a subclass of nn.Module
* finished integration test with nerfacto
* simplified config variables, integrated the parallelism/disk-data-loading all into one datamanager
* updated the splatfacto config to be simpler with the dataloading and now uses FullImageDatamanager (which has been changed)
* style checks and some cleanup
* new splatfacto test, cleaning up nerfacto integration test
* removing redundant parallel_full_images_datamaanger, as the OG full_image_datamanager now has full parallelized support
* ruff linting and pyright fixing
* further pyright fixing
* another pyright fixing
* fixing pyright error, camera optimization no longer part of datamanager
* fixing one pyright
* fixing dataloading error when camera is not undistorted with dataloader
* fixing comments and updating style
* undoing a style change i made
* undoing another style change i made by accident
* fixing slow runtime
* fixing a more general camera undistortion bug
* move images to device properly
* minor improvements
* add print statement about >500 images, cleanup method configs
* make method configs consistent across nerfacto models
* adding description comments
* updating description
* resolving some pyright issues with export.py, explained in PR desc
* fixing pyright issues in base_pipeline.py
* ran pyright on exporter and base_pipeline.py without issues
* adding a git ignore to a clearly checked pyright issue
* typo
* fixing most ns-dev-test cases
* cleanup, passing final ns-dev-test
* oops, accidentally pushed the deletion of a docstring, undoing that
* another cleanup
* some fixes to eval pipeline
* lint
* add asserts for spawn
* lint
* cleaning up import statements in parallel_datamanager.py
* adding new developer documentation if users would like to migrate their custom datamanagers to support new features
* removing unnecessary to_device no-op
* further updates to documentation
* lint
* more docs
* docs
* remove comment
* add docs, fix depth dataset with parallel datamanager, fix mask sampling bug
* remove profiling
* more profile removal
* custom_view_processor->custom_image_processor
* doc clarification
* datamanager doc nit
* whitespace
* nits
* remove stuff from __post_init__, tune num workers more, add random offset in raybatchstream
* removing unnecessary assertion, updating docstring because DataManager is no longer an nn.Module
* clarifying configuration with num_images_to_sample_from and num_times_to_repeat_images, cleaning up functions
* adding logic so that nerfacto users can load_from_disk and customize image batch sizes and repeat parameters
* ruff formatting! whoops forgot to format
* fixing logic, now if users set load_from_disk to true, datamanager will use 50 and 10. If users set it and specify their own values, we support that as well
* adding separate datamanager config so that target can be removed in method_configs
---------
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* alex/record3d_docs_add
* alex/record3d_docs_add
* link from dataparsers to custom_dataset added.
* images are now appearing
Co-authored-by: evonneng <evonne_ng@berkeley.edu>
