Making TrainerConfig an InstantiateConfig so it can be swapped in and out with different methods. Also made the TrainerConfig a subclass of ExperimentConfig (which was changed from a printableconfig to an instantiateconfig), since we were already instantiating the Trainer class with the ExperimentConfig
Removed the "config" field from the TrainerCallbackAttributes struct to eliminate circular dependencies when types are checked.
Minor scripts and docs changes to accommodate the update to the config struct, meaning the flags that were associated with the TrainerConfig were moved up to the top level of the config.
* inerted a new model configuration instant-ngp-bounded
* added more remarks and small fixes
* went back to old near plane
* added the oprtion to get a file as well as directory in nerfstudio data parser
* pretify prev commit
* review fixes
* more clear explanation in README.md
* reverted readme changes
* fixed merge bug
* review changes. update config.data docs support for json files inputs
Co-authored-by: dror <you@example.com>
This branch gives us the flexibility to use Cameras as a tensordataclass, giving arbitrary batch dimensions, slicing, etc all for free. It tests this with the phototourism dataset.
Tensordataclass changes:
Originally, the tensordataclass would look at tensors in an object and take everything in front of the last dimension to be the batch dimensions, but in the case of the camera_to_world matrices, everything except for the last TWO are the important dimensions, so we needed to have a way for the tensordataclass to acknowledge these specific fields where there are multiple data dimensions behind the batch dimensions. This is important for things like reshaping where we need to know which dimensions get reshaped and which dimensions get left alone.
Cameras changes:
Now subclass of tensordataclass, which required a change to the tensordataclass itself.
The possibility of extra batch dimensions is the hairy part that requires the majority of these changes. The extra batch dimensions aren't a hassle in every function except in ray generation. The ray generation is where this operates different relative to the old cameras. The reason for this is before, for each ray we wanted to generate, we could specify a single index into our batch of cameras, but if you have multiple camera dimensions, for each ray, you need to now specify one index per camera batch dimension, meaning our camera_indices we specify need to have one extra dimension at the end, a dimension of size Cameras.ndim (number of batch dimensions in the camera object).
I also added some neat logic in generate_rays where if you want your output raybundle to be batched, if you specify batched camera_indices, you will receive the same batch size for the output raybundle, meaning you can easily generate raybundles corresponding to things like image patches and whatnot.
Phototourism changes:
The difficulty here is that images in the phototourism dataset are of different sizes, which is difficult because we can no longer batch the image tensors. We also need to better utilize caching in the CacheDataloader since the vanilladatamanager defaults don't really use it as well as they could, and without properly utilizing it, the rays / sec are maybe 25% of the standard nerfstudio dataset speeds (which are 10% of the size of the phototourism datasets). To do this, we specify a collate function in the existing datamanager configs, and we make a subclass of the vanilladatamanagerconfig named VariableResDataManagerConfig which will have a custom collate function storing the images in a list, and using better defaults for caching in the dataloader, caching ~40 images at a time. The slowest part of using a list is pixel sampling, where we added a custom if case for if our pixel sampler receives a batch of images that are in a list, and the lack of ability to vectorize operations over a list means that caching in the dataloader is even more important.
* Add D-NeRF temporal distortion and D-NeRF model
This allows for temporal distortion to be enabled on a variety of models through config
parameters, without an explicit model. Temporal distortion and MIP-NeRF are not currently
jointly runnable but can be changed in the future.
* Simplifying changes
* Simplifying changes
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* misc cuda improvements. added cone angle as param
* wandb as default. better organization for experiments
* turn off tensorboard
* disable eval when using viewer
* remove comment
* launch.json
* testing; need to merge
* fixing wrong base file in models/base
* pushed input parser into datasets class to abstract away fetching data in dataloader
* slight name chage and all
* nits
* added dataparser
* added licnese
* added tensorf, updated requirements for windows
* cleaning up imports
* tensorf changes
* initial working version of tensorf
* added callback for feature upsampling
* modifying tests to match changes
* reverting requirements changes
* reverting unnecessary changes
* reverting unnecessary changes
* no more upsampling on step 0
* removed density field, fixed error msg
* modified callbacks and upsampling, reinitializing optimzer in training loop
* deleting tensorf_field file"
* checkpointing progress on restructuring callbacks
* updated callback logic for before and after train capabilities
* fixing upsampling stepping
* merging
* adding tensorf to config, fixing some bugs with new callback
* changed the way callbacks detected steps argument in callback func
* fixing typing
* adding pylint disable no self use to get_training_callbacks
* fixing config
* changing callback in vanilla nerf to allow for no density field
* cleanup
* removing config yaml
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
* starting docs for pipelines
* working on an example
* clearing output
* starting to implement config changes and add documentation
* wip: removed cicular imports; porting basic config into python; todo: get python setups workin
* circular imports again
* wip: pausing at dataloader sort of working
* break for today
* running vanilla
* wip need to get all the config docs working
* laptop about to die
* instant ngp working
* added all configs for all methods; not tested yet: [mipnerf360, nerfw, semantic_nerf]
* unpacking later
* done with pylinting
* mostly done. just need to do docs and one test
* added import order to docs
* fixing near/far plane and other config things
* making loggers also instantiable via configs
* docs
* lint
* making --help work with dcargs
* moving configs back into flat
* convert list to tuples; fixed post init function with mutable params
* lint
* made run_eval and benchmarking scripts compatible with new config/dcargs
* lint
* added option to load configs from presaved config
Co-authored-by: evonneng <evonne_ng@berkeley.edu>
* Model class done, pipeline class still needs more visualization stuffs
* Renaming the BaseDataContainer to DataloaderOutputs and adding documentation
* Pushing more documentation, removing some trivial functions
* Updating dict typing
* Removing config dicts for collider
* Passing tests
* Pushing shell new dataloader
* Saving work, need to fix NGP model's get_loss_dict and Pipeline
* V1 of the ngp pipeline should be done. Need train script and debugging
* Updating with latest instant-ngp code
* Syntax updates from latest ngp graph code
* Linter, removing unneeded code in new NGP graph code
* Style
* Should be good now. Linter is happy and the code is verbatum current instant-ngp
* Everything done except novel view testing code
* Fixing some nitpicks
* Debugging dataloader
* merged master
* WOP. integrating pipeline with configs and notebook demo
* removing model folder
* a lot of name changing from Graph to Model
* in progress... changing trainer to use pipeline... broken right now
* pipeline code runs
* switched to val dataset
* makes run_eval.py work
* updated configs for pipeline change
* fixing test_run_train script
* fixing black and linting
* fix docs
* small updates to trainer
Co-authored-by: Jake Austin <jake-austin@berkeley.edu>
* initial commit for viewer refactor
* updating name change to nerfactory
* adding headers
* making the rendering fix which broke from a merge
* things are starting to work. removing old commands now
* removed prints
* cleanup
* updating state works now even for nested dictionaries
* updating state in redux. listening for state changes. trying to draw three.js now
* added drawing functionality. need to fix translation of frustums and image planes
* images and frustums drawn correctly
* fixing lint
* github actions
* back to npm
* fixing eslint
* remove package.json from root
* new favicon
* some viewer cleanup
* notebook
* fixed camera drawings speed to avoid redraw
* trying to get Leva to update
* quick fix
* synced config panel
* adding notebook
* fixing output print error
