* start of mesh PR
* point cloud export mvp
* start of mesh PR
* point cloud export mvp
* Save out mesh. Poor quality atm
* reorganized mesh algorithms and specified CLI
* CLI reorg
* tsdf algorithm in progress
* debugging
* tsdf functional but camera distortion params not in
* playing with distortion params
* TSDF works but texture looks bad
* added trimesh
* moved to exporter
* removed files
* saving progress
* save texture image
* seems to work w/o barycentric
* barycentric start
* gap between triangles
* blocking out diagonal
* new barcentric and scaling
* meshing working. need to remove extra rows
* fixed triangle packing
* gitignore changes
* removed package
* fix Literal import
* comments
* update .mtl file
* poisson mesh with model output normals & resolved some PR comments on git
* starting docs. breaking up big function. todo: add xatlas
* xatlas and nvdiffrast working
* triangle_id
* progress on expensive texture coordinates method
* pytorch meshing works
* removing nvdiffrast
* remove trimesh
* cleanup
* few changes
* pred normals by default and add minimum additions to viewer
* fix cli in viewer
* add xatlas as dependency
* make point cloud more dense
* reverted viewer changes
* turn off normals by default
* fix docs
* finished todos on github
* remove xatlas
* xatlas
* black
Co-authored-by: Matt Tancik <tancik@berkeley.edu>
* Fix IndexError runtime error
fixes this error:
File "C:\code\nerfstudio\nerfstudio\data\dataparsers\nerfstudio_dataparser.py", line 232, in _generate_dataparser_outputs
distortion_params = torch.stack(distort, dim=0)[idx_tensor]
IndexError: tensors used as indices must be long, byte or bool tensors
* Update nerfstudio_dataparser.py
Per discussion at https://github.com/nerfstudio-project/nerfstudio/pull/1057#issuecomment-1332670064
* launch_eval_blender.sh: fix script
- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated
* launch_train_blender.sh: fix script
- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to https://github.com/nerfstudio-project/nerfstudio/issues/806#issuecomment-1284327844
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line
* launch_eval_blender.sh: fix script
* Update benchmarking.md
* launch_train_blender.sh: add -s to eval command-line
* Update launch_train_blender.sh
* Update benchmarking.md
* Add support for insta360 single-file format
* Improve some variable names
* Add documentation for crop_percentage parameter
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
* Added Dockerfile to build docker image including all dependencies and nerfstudio, added description on how to user docker.
* Move docker description to Installation.md.
Co-authored-by: Nicolas Zunhammer <n.zunhammer@tum.de>
This branch gives us the flexibility to use Cameras as a tensordataclass, giving arbitrary batch dimensions, slicing, etc all for free. It tests this with the phototourism dataset.
Tensordataclass changes:
Originally, the tensordataclass would look at tensors in an object and take everything in front of the last dimension to be the batch dimensions, but in the case of the camera_to_world matrices, everything except for the last TWO are the important dimensions, so we needed to have a way for the tensordataclass to acknowledge these specific fields where there are multiple data dimensions behind the batch dimensions. This is important for things like reshaping where we need to know which dimensions get reshaped and which dimensions get left alone.
Cameras changes:
Now subclass of tensordataclass, which required a change to the tensordataclass itself.
The possibility of extra batch dimensions is the hairy part that requires the majority of these changes. The extra batch dimensions aren't a hassle in every function except in ray generation. The ray generation is where this operates different relative to the old cameras. The reason for this is before, for each ray we wanted to generate, we could specify a single index into our batch of cameras, but if you have multiple camera dimensions, for each ray, you need to now specify one index per camera batch dimension, meaning our camera_indices we specify need to have one extra dimension at the end, a dimension of size Cameras.ndim (number of batch dimensions in the camera object).
I also added some neat logic in generate_rays where if you want your output raybundle to be batched, if you specify batched camera_indices, you will receive the same batch size for the output raybundle, meaning you can easily generate raybundles corresponding to things like image patches and whatnot.
Phototourism changes:
The difficulty here is that images in the phototourism dataset are of different sizes, which is difficult because we can no longer batch the image tensors. We also need to better utilize caching in the CacheDataloader since the vanilladatamanager defaults don't really use it as well as they could, and without properly utilizing it, the rays / sec are maybe 25% of the standard nerfstudio dataset speeds (which are 10% of the size of the phototourism datasets). To do this, we specify a collate function in the existing datamanager configs, and we make a subclass of the vanilladatamanagerconfig named VariableResDataManagerConfig which will have a custom collate function storing the images in a list, and using better defaults for caching in the dataloader, caching ~40 images at a time. The slowest part of using a list is pixel sampling, where we added a custom if case for if our pixel sampler receives a batch of images that are in a list, and the lack of ability to vectorize operations over a list means that caching in the dataloader is even more important.
* Updating the script for downloading data
* Making update to config structure
* Updating documentation and colab
* Adding back the save directory
* Pushing an all flag for nerfstudio
* forgot to save some previous edits
* add support for per-image mask filenames in transforms.json
* pass None instead of empty list to DataParserOutputs constructor
* look for downsampled masks in correct folder
AABB normalized_positions to [-1, 1]
Previously, SceneBox.get_normalized_positions() returned a value in 0,1.
This was fine for InstantNGP and Nerfacto, which use these values directly in an MLP.
For TensoRF though, this was an error because grid sample expects values in the range [-1,1].
Thus, I think only 1/4th of the encoding planes were actually being used, and half of the
encoding lines were.
* staring normals PR
* code runs but visualization looks bad
* Camera and world coordinate conventions (#889)
* documented coordinate conventions without a figure for now
* another sentence
* update frames docs
* explain intrinsics and extrinsics
* more comments
* fix build
* Start viewer with more accurate speed calculation (#899)
Start viewer with more accurate speed calc
* Minor doc updates (#903)
* fix conflicts
* code runs but visualization looks bad
* normals implemented and vis fixed. still debugging quality
* minor name change
* comment
* fix black
* lint
* predicting normals
* adding detach
* predicted normals
* improved normals prediction performance
* cleaning code
* cleaning code
* cleaning code
* adding type to function param
* fixing tensorf forward func
* reverting tensorf changes
* fix style with tensorf field
* adding normals check to tensorffield
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: terrance <terrance.3512@gmail.com>
Ethan Weber
Ethan Weber
machenmusik
Alexander DemidkoandAlexander Demidko
Brent Yi
Mustafa Berk YALDIZ
hturki
Kevin Chen
Julian Knodt
Frédéric Devernay
cnsumnerandMatthew Tancik
Nicolas ZunhammerandNicolas Zunhammer
David McAllister
Zak Murezandzmurez
Alexander Kristoffersen
Jake Austin
Daniel Crispell
ksnzh
Nikhil Mohan