Anthony Zhang 4a3e3e6403 Dataloading Revamp (#3216) ... * initial debugging and testing works * pwais changes with RayBatchStream to alleviate training * few bugs to iron out with multiprocessing, specifically pickled collate_fn * working version of RayBatchStream * additional docstrings * cleanup * much more documentation * successfully trained AEA-script2_seq2 closed_loop without OOM * porting over aria dataset-size feature * added logic to handle eviction of a worker's cached_collated_batch * antonio's implementation of stream batches * training on a dataset with 4000 images works! * some configuration speedups, loops aren't actually needed! * quick fix adjustment to aria * removed unnecessary looping * much faster training when adding i variable to collate every 5 ray bundles * cleanup unnecssary variables in Dataloader * further cleanup * adding caching of compressed images to RAM to reduce disk bottleneck * added caching to RAM for masks * found fast way to collate - many tricks applied * quick update to aria to test on different datasets * cleaned up the accelerated pil_to_numpy function * cleaning up PR * this commit was used to generate the time metrics and profiling metrics * REAL commit used to run tests * testing with nerfacto-big * generated RayBundle collate and converting images from uint8s to float32 on GPU tests * updating nerfacto to support uint8 easily, will need to figure out a way to contain this within the datamanager API * datamanager updates, both splat and nerf * must use writeable arrays because torch requires them * cleaned up base_dataset, added pickle to utils, more code in full_image, and cleaner desc for base_datamanager * lots of process on a parallel FullImageDatamanger * can train big splats with pre-assertion hack or ROI hack and 0 workers * fixed all undistortion issues with ParallelImageDatamanager * adding some downsampling and parallel tests with splatfacto! * deleted commented code in dataloaders.py and added bugfix to shuffling * testing splatfacto-big * cleaned up base_pipeline.py * cleaned up base_pipeline.py ACTUALLY THIS TIME, forgot to save last time * cleaned up a lot of code * process_project_aria back to main branch and some cleanup in full_image_datamanager * clarifying docstrings * further PR cleanup * updating models * further cleanup * removed caching of images into bytestrings * adding caching of compressed images to RAM, forgot that hardware matters * removing oom methods, adding the ability to add a flag to dataloading * removed CacheDataloader, moved RayBatchStream to dataloaders.py, new vanilla_datamanager rewritten * fixing base_piplines, deleting a weird datamanager_configs file that was accidently created * cleaning up next_train * replaced parallel datamanager with new datamanager * reverted the original base_datamanager.py, new datamanager replaced parallel_datamanager.py * modified VanillaConfig, but VanillaDataManager is the same as before * cleaning up, 2 datamanagers now - original and new parallel one * able to train with new nerfstudio dataloader now * side by side datamanagers, moved tons of logic into dataloaders.py and created new files for our parallel datamangers * added custom ray processing API to support implementations like LERF, cleaned up FullImageDatamanager to original because of new ParallelImageDatamanger * adding functionality for ns-eval by adding FixedIndicesEvalDataloader to the setup_eval * adding both ray API and image-view API to datamanagers for custom parallelization * updating splatfacto config for 4k tests * updating docstrings to be more descriptive * new datamanager API breaks when setup_eval() has multiple workers, not sure why but single worker will have to do * adding custom_view_processor to ImageBatchStream * reverting full_images_datamanager to main branch * removing nn.Module inheritance from Datamanager class * don't need to move datamanger to device anymore since Datamanager is not a subclass of nn.Module * finished integration test with nerfacto * simplified config variables, integrated the parallelism/disk-data-loading all into one datamanager * updated the splatfacto config to be simpler with the dataloading and now uses FullImageDatamanager (which has been changed) * style checks and some cleanup * new splatfacto test, cleaning up nerfacto integration test * removing redundant parallel_full_images_datamaanger, as the OG full_image_datamanager now has full parallelized support * ruff linting and pyright fixing * further pyright fixing * another pyright fixing * fixing pyright error, camera optimization no longer part of datamanager * fixing one pyright * fixing dataloading error when camera is not undistorted with dataloader * fixing comments and updating style * undoing a style change i made * undoing another style change i made by accident * fixing slow runtime * fixing a more general camera undistortion bug * move images to device properly * minor improvements * add print statement about >500 images, cleanup method configs * make method configs consistent across nerfacto models * adding description comments * updating description * resolving some pyright issues with export.py, explained in PR desc * fixing pyright issues in base_pipeline.py * ran pyright on exporter and base_pipeline.py without issues * adding a git ignore to a clearly checked pyright issue * typo * fixing most ns-dev-test cases * cleanup, passing final ns-dev-test * oops, accidentally pushed the deletion of a docstring, undoing that * another cleanup * some fixes to eval pipeline * lint * add asserts for spawn * lint * cleaning up import statements in parallel_datamanager.py * adding new developer documentation if users would like to migrate their custom datamanagers to support new features * removing unnecessary to_device no-op * further updates to documentation * lint * more docs * docs * remove comment * add docs, fix depth dataset with parallel datamanager, fix mask sampling bug * remove profiling * more profile removal * custom_view_processor->custom_image_processor * doc clarification * datamanager doc nit * whitespace * nits * remove stuff from __post_init__, tune num workers more, add random offset in raybatchstream * removing unnecessary assertion, updating docstring because DataManager is no longer an nn.Module * clarifying configuration with num_images_to_sample_from and num_times_to_repeat_images, cleaning up functions * adding logic so that nerfacto users can load_from_disk and customize image batch sizes and repeat parameters * ruff formatting! whoops forgot to format * fixing logic, now if users set load_from_disk to true, datamanager will use 50 and 10. If users set it and specify their own values, we support that as well * adding separate datamanager config so that target can be removed in method_configs --------- Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com> Co-authored-by: Brent Yi <yibrenth@gmail.com>		2025-01-31 10:40:01 -08:00
..
imgs
datamanagers.md
dataparsers.md
fields.md
index.rst
models.md
pipelines.md