The density and alpha compositing helpers are annotated for arbitrary leading ray batch dimensions, but create identity prefixes from only the first batch axis. The alpha helper also accumulates and slices along dimension 1. Unbatched rays and image-shaped ray batches consequently fail or composite along the wrong axis.
* fix core tests
* Try alternative fix for pyright errors
* Run ruff, bump core code checks to 3.11
* Run correct ruff version
* Use the previous pyright version 👀
* cv2 ignore, pin dev dependency to newer pytorch version
* ruff
* Remove pyright error suppressions
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* sample pixels from all data in variable-resolution batches
* sort import block
* sort imports
* reorder imports
* clean up warnings
* simplify index extraction
* fixing ns-eval when datamanager is set to disk for splats
* fixing prefetch_factor type issues if user decides to use 0 workers
* removing accidental file
* fixing depth nerfacto config for issue 3592
* ruff linting the imports
* fixing workers creating GPU tensors serialization error
* finished next_eval
* adding flowcharts!
* fixing ruff
* added flowcharts into datamanagers.md
* simplying code and adding GPU serialization fix for both datamanagers
* fixing nits with consistent styling
* last optional nit
* fixing num workers
* initial debugging and testing works
* pwais changes with RayBatchStream to alleviate training
* few bugs to iron out with multiprocessing, specifically pickled collate_fn
* working version of RayBatchStream
* additional docstrings
* cleanup
* much more documentation
* successfully trained AEA-script2_seq2 closed_loop without OOM
* porting over aria dataset-size feature
* added logic to handle eviction of a worker's cached_collated_batch
* antonio's implementation of stream batches
* training on a dataset with 4000 images works!
* some configuration speedups, loops aren't actually needed!
* quick fix adjustment to aria
* removed unnecessary looping
* much faster training when adding i variable to collate every 5 ray bundles
* cleanup unnecssary variables in Dataloader
* further cleanup
* adding caching of compressed images to RAM to reduce disk bottleneck
* added caching to RAM for masks
* found fast way to collate - many tricks applied
* quick update to aria to test on different datasets
* cleaned up the accelerated pil_to_numpy function
* cleaning up PR
* this commit was used to generate the time metrics and profiling metrics
* REAL commit used to run tests
* testing with nerfacto-big
* generated RayBundle collate and converting images from uint8s to float32 on GPU tests
* updating nerfacto to support uint8 easily, will need to figure out a way to contain this within the datamanager API
* datamanager updates, both splat and nerf
* must use writeable arrays because torch requires them
* cleaned up base_dataset, added pickle to utils, more code in full_image, and cleaner desc for base_datamanager
* lots of process on a parallel FullImageDatamanger
* can train big splats with pre-assertion hack or ROI hack and 0 workers
* fixed all undistortion issues with ParallelImageDatamanager
* adding some downsampling and parallel tests with splatfacto!
* deleted commented code in dataloaders.py and added bugfix to shuffling
* testing splatfacto-big
* cleaned up base_pipeline.py
* cleaned up base_pipeline.py ACTUALLY THIS TIME, forgot to save last time
* cleaned up a lot of code
* process_project_aria back to main branch and some cleanup in full_image_datamanager
* clarifying docstrings
* further PR cleanup
* updating models
* further cleanup
* removed caching of images into bytestrings
* adding caching of compressed images to RAM, forgot that hardware matters
* removing oom methods, adding the ability to add a flag to dataloading
* removed CacheDataloader, moved RayBatchStream to dataloaders.py, new vanilla_datamanager rewritten
* fixing base_piplines, deleting a weird datamanager_configs file that was accidently created
* cleaning up next_train
* replaced parallel datamanager with new datamanager
* reverted the original base_datamanager.py, new datamanager replaced parallel_datamanager.py
* modified VanillaConfig, but VanillaDataManager is the same as before
* cleaning up, 2 datamanagers now - original and new parallel one
* able to train with new nerfstudio dataloader now
* side by side datamanagers, moved tons of logic into dataloaders.py and created new files for our parallel datamangers
* added custom ray processing API to support implementations like LERF, cleaned up FullImageDatamanager to original because of new ParallelImageDatamanger
* adding functionality for ns-eval by adding FixedIndicesEvalDataloader to the setup_eval
* adding both ray API and image-view API to datamanagers for custom parallelization
* updating splatfacto config for 4k tests
* updating docstrings to be more descriptive
* new datamanager API breaks when setup_eval() has multiple workers, not sure why but single worker will have to do
* adding custom_view_processor to ImageBatchStream
* reverting full_images_datamanager to main branch
* removing nn.Module inheritance from Datamanager class
* don't need to move datamanger to device anymore since Datamanager is not a subclass of nn.Module
* finished integration test with nerfacto
* simplified config variables, integrated the parallelism/disk-data-loading all into one datamanager
* updated the splatfacto config to be simpler with the dataloading and now uses FullImageDatamanager (which has been changed)
* style checks and some cleanup
* new splatfacto test, cleaning up nerfacto integration test
* removing redundant parallel_full_images_datamaanger, as the OG full_image_datamanager now has full parallelized support
* ruff linting and pyright fixing
* further pyright fixing
* another pyright fixing
* fixing pyright error, camera optimization no longer part of datamanager
* fixing one pyright
* fixing dataloading error when camera is not undistorted with dataloader
* fixing comments and updating style
* undoing a style change i made
* undoing another style change i made by accident
* fixing slow runtime
* fixing a more general camera undistortion bug
* move images to device properly
* minor improvements
* add print statement about >500 images, cleanup method configs
* make method configs consistent across nerfacto models
* adding description comments
* updating description
* resolving some pyright issues with export.py, explained in PR desc
* fixing pyright issues in base_pipeline.py
* ran pyright on exporter and base_pipeline.py without issues
* adding a git ignore to a clearly checked pyright issue
* typo
* fixing most ns-dev-test cases
* cleanup, passing final ns-dev-test
* oops, accidentally pushed the deletion of a docstring, undoing that
* another cleanup
* some fixes to eval pipeline
* lint
* add asserts for spawn
* lint
* cleaning up import statements in parallel_datamanager.py
* adding new developer documentation if users would like to migrate their custom datamanagers to support new features
* removing unnecessary to_device no-op
* further updates to documentation
* lint
* more docs
* docs
* remove comment
* add docs, fix depth dataset with parallel datamanager, fix mask sampling bug
* remove profiling
* more profile removal
* custom_view_processor->custom_image_processor
* doc clarification
* datamanager doc nit
* whitespace
* nits
* remove stuff from __post_init__, tune num workers more, add random offset in raybatchstream
* removing unnecessary assertion, updating docstring because DataManager is no longer an nn.Module
* clarifying configuration with num_images_to_sample_from and num_times_to_repeat_images, cleaning up functions
* adding logic so that nerfacto users can load_from_disk and customize image batch sizes and repeat parameters
* ruff formatting! whoops forgot to format
* fixing logic, now if users set load_from_disk to true, datamanager will use 50 and 10. If users set it and specify their own values, we support that as well
* adding separate datamanager config so that target can be removed in method_configs
---------
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
* limited the tyro version up to 0.9.2
* bump tyro>=0.9.8
* Bump typeguard
* Runtime checking with jaxtyping hook instead of typeguard
* Fix test for Python >=3.11, caused by missing default_factory
---------
Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
* feat: improve launch device
* add licence header
* crash instead of trying to set the available option
* set cuda as default
---------
Co-authored-by: Noé Pion <noe.pion@onfido.com>
Co-authored-by: J.Y. <132313008+jb-ye@users.noreply.github.com>
* Move util functions out of splatfacto
Nothing else currently uses some of the SH utils, but it might make sense to get them out of splatfacto.
I also moved the k nearest neighbors to utils since it doesn't depend on the model class.
* fix assert
* fix sh test
* convert to using degrees, not levels
Fix: number of iterations wasn't set correctly
The original code causes iteration to go over max_num_iterations, causing progress tracker be unable to estimate time needed to finish, and causes unwanted extra iterations
Afloat
couperle
Qizhi Chen
Frédéric DevernayandBrent Yi
Emmanuel Ferdman
Daniel Crispell
Anthony Zhang
HeaInandBrent Yi
klimermanaandBrent Yi
Dávid Komorowicz
beckyfeng08
Brian XuandJonáš Kulhánek
Yudong JinandJ.Y.
Tim de Jager
Roger Q.
Jameson Crate
clonefetchandAlexander Kristoffersen
Alexander Kristoffersen
Qizhi Chen
Daniel Wang
Kevin Chen
Lei Lei
SebastiaanandJ.Y.
Simon Bethke
Nicholas Pfaff
Aayush GuptaandJ.Y.
