* started adding equirectangular to process_data
* was multiplying frame by 255 twice
* added equi2planar configs for the 3 presets
* fixed presets
* setup process data
* changed variable name and comment
* fixed middle preset
* removed third preset
* added progress bar and fixed video for equirectangular input
* fixed whitespace stuff
* added pyequilib dependencey
* Updated for JPG support for equirectangular image input
Updated process_data_utils for JPG support for equirectangular image input, since even though .jpg images worked, .JPG images (output from 360 camera software) were not working.
* changed file type matching
* added equirect utils
* added license header
* added docstring on equirect utils
* added heuristic for image size
* added heuristic for videos
* fixed linty stuff
* shortened code
* raised error if no ims in directory
* Updated ProcessVideo for fixing directories for equirectangular input
Updated the ProcessVideo for equirectangular input to make the image directory have the correct perspective projections. The original equirectangular images are copied into a temp_images folder which also holds the perspective projections. This folder is deleted once the perspective projections are copied into the process data images folder.
---------
Co-authored-by: cvachha <cyrus.vachha@yahoo.com>
Co-authored-by: cvachha <cvachha549@gmail.com>
* Allow black/white background color in Nerfacto
* Use white background in Nerfacto Blender Benchmark
* Fixed unused import
* Updated index for license headers script
* Allow skipping copy+downscale of images in ns-process-data, fix bug when using --skip-colmap on images.bin with non-standard image names
* Rename flags and improve error messages
---------
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Updated Blender on calculating duration
Updated how the Blender add on calculates duration with frame steps and added the case when frame step size is 0.
* Blender addon script and documentation
First version of blender addon python script located in scripts/blender and new documentation for the addon. The Blender addon is for importing or exporting a camera path for visual effects and compositing.
* Updated formatting of Blender script
Updated the formatting based on the script tests for the blender addon script.
* Updated Blender addon to fix FOV and alignment
Updated how FOV is calculated for both blender cam to json and json to blender cam. The FOV is calculated based on the aspect ratio, angle of view, and sensor fit and size.
* Updated Blender addon documentation with examples
Updated the Blender add-on documentation with a few example gif composite animations
* Updated Blender addon to match frame rate and fixed formatting
Updated the Blender add-on script and docs to match the Blender frame rate. Also the formatting should be fixed. There are also now error message popups instead of console prints for invalid input in the add-on.
* Updated the blender addon for formatting
Updated pylint formatting and comments
* implement the camera exporter
* remove train_dataset from exporter
* fix linter errors
* add docstring
* correct comment
* fix linter errors
* make image_filenames a property of the dataset
Making TrainerConfig an InstantiateConfig so it can be swapped in and out with different methods. Also made the TrainerConfig a subclass of ExperimentConfig (which was changed from a printableconfig to an instantiateconfig), since we were already instantiating the Trainer class with the ExperimentConfig
Removed the "config" field from the TrainerCallbackAttributes struct to eliminate circular dependencies when types are checked.
Minor scripts and docs changes to accommodate the update to the config struct, meaning the flags that were associated with the TrainerConfig were moved up to the top level of the config.
* use max resolution when not moving or training
* work around mismatch to allow rgb depth side-by-side output
* Update render.py
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* when camera_type is equirectangular, add spherical metadata to rendered video
* remove unnecessary CONSOLE.print
* refactor to make pylint happier
* get os.stat size properly; document camera_type argument
* write video on the fly to avoid holding all images in memory
Note: video render width and height were unspecified for spiral trajectory; this uses values for Nerfacto
* fix images output
* doc fix
* inerted a new model configuration instant-ngp-bounded
* added more remarks and small fixes
* went back to old near plane
* added the oprtion to get a file as well as directory in nerfstudio data parser
* pretify prev commit
* review fixes
* more clear explanation in README.md
* reverted readme changes
* fixed merge bug
* review changes. update config.data docs support for json files inputs
Co-authored-by: dror <you@example.com>
* basic support for hloc (includes netvlad, super point, superglue in colmap), just a first test, not ready yet
* add exhaustive match option for hloc
* add command line option to choose the type of feature, matcher,...
* make hloc an optional dependency, update doc, and small bug fixes
* fix missing imports
* fix warnings, pylint,...
* fix overwrite previous commits
* fix import order from isort
* move find_tool_feature_matcher_combination to process_data_utils, add type hints and Args/Returns doc
* error message when no valid combination of sfm tool, feature type, and matcher
Co-authored-by: RandomPrototypes <delmotte_ar@yahoo.fr>
* start of mesh PR
* point cloud export mvp
* start of mesh PR
* point cloud export mvp
* Save out mesh. Poor quality atm
* reorganized mesh algorithms and specified CLI
* CLI reorg
* tsdf algorithm in progress
* debugging
* tsdf functional but camera distortion params not in
* playing with distortion params
* TSDF works but texture looks bad
* added trimesh
* moved to exporter
* removed files
* saving progress
* save texture image
* seems to work w/o barycentric
* barycentric start
* gap between triangles
* blocking out diagonal
* new barcentric and scaling
* meshing working. need to remove extra rows
* fixed triangle packing
* gitignore changes
* removed package
* fix Literal import
* comments
* update .mtl file
* poisson mesh with model output normals & resolved some PR comments on git
* starting docs. breaking up big function. todo: add xatlas
* xatlas and nvdiffrast working
* triangle_id
* progress on expensive texture coordinates method
* pytorch meshing works
* removing nvdiffrast
* remove trimesh
* cleanup
* few changes
* pred normals by default and add minimum additions to viewer
* fix cli in viewer
* add xatlas as dependency
* make point cloud more dense
* reverted viewer changes
* turn off normals by default
* fix docs
* finished todos on github
* remove xatlas
* xatlas
* black
Co-authored-by: Matt Tancik <tancik@berkeley.edu>
* launch_eval_blender.sh: fix script
- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated
* launch_train_blender.sh: fix script
- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to https://github.com/nerfstudio-project/nerfstudio/issues/806#issuecomment-1284327844
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line
* launch_eval_blender.sh: fix script
* Update benchmarking.md
* launch_train_blender.sh: add -s to eval command-line
* Update launch_train_blender.sh
* Update benchmarking.md
* Add support for insta360 single-file format
* Improve some variable names
* Add documentation for crop_percentage parameter
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
This branch gives us the flexibility to use Cameras as a tensordataclass, giving arbitrary batch dimensions, slicing, etc all for free. It tests this with the phototourism dataset.
Tensordataclass changes:
Originally, the tensordataclass would look at tensors in an object and take everything in front of the last dimension to be the batch dimensions, but in the case of the camera_to_world matrices, everything except for the last TWO are the important dimensions, so we needed to have a way for the tensordataclass to acknowledge these specific fields where there are multiple data dimensions behind the batch dimensions. This is important for things like reshaping where we need to know which dimensions get reshaped and which dimensions get left alone.
Cameras changes:
Now subclass of tensordataclass, which required a change to the tensordataclass itself.
The possibility of extra batch dimensions is the hairy part that requires the majority of these changes. The extra batch dimensions aren't a hassle in every function except in ray generation. The ray generation is where this operates different relative to the old cameras. The reason for this is before, for each ray we wanted to generate, we could specify a single index into our batch of cameras, but if you have multiple camera dimensions, for each ray, you need to now specify one index per camera batch dimension, meaning our camera_indices we specify need to have one extra dimension at the end, a dimension of size Cameras.ndim (number of batch dimensions in the camera object).
I also added some neat logic in generate_rays where if you want your output raybundle to be batched, if you specify batched camera_indices, you will receive the same batch size for the output raybundle, meaning you can easily generate raybundles corresponding to things like image patches and whatnot.
Phototourism changes:
The difficulty here is that images in the phototourism dataset are of different sizes, which is difficult because we can no longer batch the image tensors. We also need to better utilize caching in the CacheDataloader since the vanilladatamanager defaults don't really use it as well as they could, and without properly utilizing it, the rays / sec are maybe 25% of the standard nerfstudio dataset speeds (which are 10% of the size of the phototourism datasets). To do this, we specify a collate function in the existing datamanager configs, and we make a subclass of the vanilladatamanagerconfig named VariableResDataManagerConfig which will have a custom collate function storing the images in a list, and using better defaults for caching in the dataloader, caching ~40 images at a time. The slowest part of using a list is pixel sampling, where we added a custom if case for if our pixel sampler receives a batch of images that are in a list, and the lack of ability to vectorize operations over a list means that caching in the dataloader is even more important.
* Updating the script for downloading data
* Making update to config structure
* Updating documentation and colab
* Adding back the save directory
* Pushing an all flag for nerfstudio
* forgot to save some previous edits
