Commit Graph
42 Commits
Author SHA1 Message Date
Nicholas Pfaff 56556d22fa Rejection sample mask for patch pixel sampler (#3457)
Use mask rejection sampling in PatchPixelSampler
2024-10-01 09:54:28 -07:00
Nicholas Pfaff 4c7297c75a Speed up PairPixelSampler (#3452) 2024-09-30 17:31:49 -07:00
J.Y. 702886b213 fix ruff check (#3328) 2024-07-25 09:22:59 +09:00
c17d1d7315 Add kwargs handling for patchpixelsampler patch size (#2622)
* Add kwargs handling for patchpixelsampler patch size

* Run black

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: J.Y <132313008+jb-ye@users.noreply.github.com>
2024-05-08 10:45:58 -07:00
Carl Lindström 759fda8285 Fix bug in pixel sampler (#3103)
* Fix bug in pixel sampler

* Refactor code for distributing rays across images
2024-04-30 09:57:20 -04:00
Ethan WeberandJustin Kerr 0555132d40 Fix depth-nerfacto edge cases (#2939)
* fixes pixel sampling

* reverting indexing change

* adressing PR change

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-03-03 11:10:17 -08:00
Brent Yi c491e3e101 Fix Nerfacto + Aria (#2950) 2024-02-26 18:16:54 +00:00
Ethan WeberandJustin Kerr c397588300 Mask flexibility for methods than need to sample in the masked region (#2647)
* minor changes to make masking more flexible

* formatting

* updating ignore mask

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-29 11:47:49 -08:00
a78ca29470 Speed up mask sampling with rejection sampling (#2585)
* change masked pixel sampling to use rejection sampling instead of torch.nonzero

* black reformat code

* pyright unbound variable num_valid

* pyright type issues with num_valid

* add configuration settings for rejection sampling masks

* black reformat

* maybe this fixes it?

* revert behavior if mask sampling failed, still raise warning

* on iteration failure, use non-rejection sampling to generate indices

* ruff

---------

Co-authored-by: adrian_chang <adrian.chang@visionsystemsinc.com>
Co-authored-by: Alexander Kristoffersen <akristoffersen@berkeley.edu>
2024-01-18 18:21:44 +00:00
Matthew Hoffman f0aadf9de8 Use ruff as linter and formatter (#2765)
* Switch from black to ruff format

* Sort dependencies with ruff check

* Run ruff format
2024-01-15 13:43:14 -08:00
Chris SweeneyandBrent Yi 6ccbc2de45 Add support for Project Aria datasets. (#2617)
* Add support for Project Aria datasets.

1) An export script for processing Project Aria datasets
2) Add support for Fisheye624 cameras
3) Add support for center-crop masking and sampling

Example commands for running nerfstudio on Project Aria data:

python scripts/datasets/process_project_aria.py --vrs_file=<path_to_vrs> --mps_data_dir=<path to MPS directory> --output_dir=<nerfstudio_data_dir>

ns-train nerfacto --data <nerfstudio_data_dir>  nerfstudio-data --orientation-method none

* Clean up type errors

* Fix formatting

* Fix linter errors, import, and use absolute file paths

* Add aria subcommand to ns-process-data

* Appease ruff

* Add Aria to docs

* Fix docs, add back import try/except

* nit: path.as_posix() -> str(path)

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-12 22:18:13 +00:00
0cb410001b Fixes bug in PairPixelSampler when working with masked dataset (#2368)
* Fixes bug in PairPixelSampler when working with masked dataset

* Formatting

* Black formatting now ok.

---------

Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-01 01:11:11 +00:00
Justin Kerr e694f341a7 Move camera optimization out of datamanager and parallelize dataloading (#2092) 2023-10-10 14:52:25 -07:00
pierremerriaux-leddartech 7856aa0060 fix:correction of PairPixelSampler in case of using masks with depth-nerfacto (#2477)
fix:correction of PairPixelSampler in case of using masks
2023-09-29 16:49:20 -07:00
anc2001andBrent Yi 805e6e702d fix pixel sampling with masks issue when data is list (#2369)
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-09-14 16:11:15 +00:00
elenacliuandAdamRashid96 ae5d569376 Fix: typo in PairPixelSamplerConfig's description (#2372)
Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
2023-08-26 03:09:44 -07:00
6e37b2b542 Enable SparseNerf depth ranking loss to be used with GT/Sensor depth (#2312)
* add ranking loss to depth-nerfacto with GT depth

* formating

* pair pixel sampler bug fix from pr #2315

---------

Co-authored-by: Sasha Sax <asax@stanford.edu>
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: AdamRashid96 <abrashid@berkeley.edu>
Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
2023-08-26 02:56:52 -07:00
Sasha SaxandMatthew Tancik 9cd957f776 Update pixel_samplers.py (#2327)
* Update pixel_samplers.py

Now uses the specified batch_size param in sample_method()

* black

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-08-17 18:04:51 +00:00
Ethan Weberandmachenmusik 9b6010ea0d Adding support for PairPixelSampler and PatchPixelSampler masks (#2305)
* changes

* pair pixel sampling with a mask

* reverting changes

* revert changes

* remove line in erode

* support for patch pixel samplerr masks

* fix

---------

Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
2023-08-05 10:40:23 -04:00
7e90cb7989 PixelSampler Fixes (#2298)
* monodepth implemented

* device

* pair pixel sampler

* docstring

* removing unused imports

* fix bugs

* cache in cache dir, no longer edit json file

* moving to depth nerfacto

* fixed bug using median depth by accident

* removing depthdataset from nerfacto

* added pixelsamplerconfig

* added equirectangular warning back

* bug fix

* delete sample_config.py + small fixes

* rename to pixel_sampler

* cleaning

* test

* passing num_rays_per_batch properly

* fixed transforms typo

* renaming in nerfacto

* remove depth files from test data

* removed defaulting to base pixel sampler with mask

* adding patch_size back

* Warning for notimplemented error and patch_size fix

* fix config.pixel_sampler check

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
2023-08-04 10:27:35 -04:00
71c6b9cf3d Monodepth Regularization (#2284)
* monodepth implemented

* device

* pair pixel sampler

* docstring

* removing unused imports

* fix bugs

* cache in cache dir, no longer edit json file

* moving to depth nerfacto

* fixed bug using median depth by accident

* removing depthdataset from nerfacto

* added pixelsamplerconfig

* added equirectangular warning back

* bug fix

* delete sample_config.py + small fixes

* rename to pixel_sampler

* cleaning

* test

* passing num_rays_per_batch properly

* fixed transforms typo

* renaming in nerfacto

* remove depth files from test data

* removed defaulting to base pixel sampler with mask

* adding patch_size back

---------

Co-authored-by: AdamRashid96 <abrashid@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-08-01 18:52:47 -07:00
Milan van Wouden d413ca4204 fix large number of typos in repository (#1977)
* fix large number of typos in repository

* fix typos in tests
2023-05-23 10:41:49 -07:00
Lukas Hoellein d8f2446122 Fix mask check in PatchPixelSampler (#1971)
* fix mask check in PatchPixelSampler

* change torch.Tensor to Tensor
2023-05-22 10:43:14 -07:00
Brent YiandMatt Tancik d542646c72 Switch to ruff (#1939)
* `isort` -> `ruff`

* `pylint` -> `ruff`

* Remove line

* More minor cleanup

* Better documentation for rules we are enabling

* Fix gitignore, run black

* vscode settings

---------

Co-authored-by: Matt Tancik <mattt4001@gmail.com>
2023-05-17 10:08:14 -07:00
Brent YiandMatt Tancik 6ba2a18b2b torchtyping => jaxtyping (#1838)
* Run `ruff --fix`

* Add jaxtyping migration script

* Run migration scripts + ruff + black

* Merge remote-tracking branch 'origin/main' into brent/jaxtyping

* black

* Fix lint

* fix lint

* Cleanup and more specific types

* Fixed annotation errors

* Fix more errors

* Fix annotation

* Delete _torchtyping_to_jaxtyping.py

---------

Co-authored-by: Matt Tancik <mattt4001@gmail.com>
2023-05-16 15:57:47 -07:00
Ethan Weber 13e0fce122 Update copyright (#1917)
* updated copyright

* including txt file

* updated black

* removing license

* fixing license
2023-05-12 16:16:13 -07:00
Brent Yi 63c6303736 Bump minimum Python version to 3.8 (#1876)
* Bump minimum Python version to 3.8

* typing_extensions.Literal -> typing.Literal

* ns-dev-test

* typing_extensions.get_args => typing.get_args

* Format with older version of black

* Bump `black`

* Run black
2023-05-06 13:29:48 -07:00
Matias Turkulainen ea35897668 Remove unecessary assert in pixel sampler on image dimensions (#1861)
remove uneccessary assert on image dimension
2023-05-03 09:28:11 -07:00
Yuantao Chenandmachenmusik d875f5f5b5 fix_mask (#1819)
fix_nerfacto_mask

Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
2023-04-25 21:32:20 -04:00
Alexander Kristoffersen 99afaa7164 Patch Sampling bug fix (#1578)
quick fix.
2023-03-09 17:30:41 -08:00
Alexander Kristoffersen f8e8d8266e Patch Sampling (#1458)
* patch sampling.

* changed str to literal.

* clean up sample selector.
2023-02-19 22:14:27 -08:00
Ethan Weber 642a9e00f0 Quick fix for mask None check (#1193)
quick fix
2022-12-31 19:29:26 -07:00
Alexander Kristoffersen fd33a1e950 PixelSampler refactor (#1163)
* moved collate methods as class methods, so sampling is only thing changed.

* disable pylint error, expected to be overrided.

* update.

* more pylint.

* final lint.

* cleanup, mask fix.
2022-12-23 00:49:28 -08:00
JamesPerlman 7fe4a77b96 fix typo: "dictioary" -> "dictionary" (#1162) 2022-12-22 11:00:47 -07:00
Tao 1a70665e76 Fix slow training issue when using mask (#1114) 2022-12-13 10:16:34 -08:00
hturki 8c3f093ecb Fix device mismatch error when using masks (#1061)
* Fix device mismatch error when using masks

* Update pixel_samplers.py
2022-11-30 23:06:23 -08:00
Kevin Chen 87c3ead680 Add Support for Equirectangular Images (#966) 2022-11-28 21:29:02 -08:00
Ethan Weber b23d76cc47 Cameras TensorDataclass and Phototourism (#961)
This branch gives us the flexibility to use Cameras as a tensordataclass, giving arbitrary batch dimensions, slicing, etc all for free. It tests this with the phototourism dataset.

Tensordataclass changes:
Originally, the tensordataclass would look at tensors in an object and take everything in front of the last dimension to be the batch dimensions, but in the case of the camera_to_world matrices, everything except for the last TWO are the important dimensions, so we needed to have a way for the tensordataclass to acknowledge these specific fields where there are multiple data dimensions behind the batch dimensions. This is important for things like reshaping where we need to know which dimensions get reshaped and which dimensions get left alone.

Cameras changes:
Now subclass of tensordataclass, which required a change to the tensordataclass itself.
The possibility of extra batch dimensions is the hairy part that requires the majority of these changes. The extra batch dimensions aren't a hassle in every function except in ray generation. The ray generation is where this operates different relative to the old cameras. The reason for this is before, for each ray we wanted to generate, we could specify a single index into our batch of cameras, but if you have multiple camera dimensions, for each ray, you need to now specify one index per camera batch dimension, meaning our camera_indices we specify need to have one extra dimension at the end, a dimension of size Cameras.ndim (number of batch dimensions in the camera object).
I also added some neat logic in generate_rays where if you want your output raybundle to be batched, if you specify batched camera_indices, you will receive the same batch size for the output raybundle, meaning you can easily generate raybundles corresponding to things like image patches and whatnot.

Phototourism changes:
The difficulty here is that images in the phototourism dataset are of different sizes, which is difficult because we can no longer batch the image tensors. We also need to better utilize caching in the CacheDataloader since the vanilladatamanager defaults don't really use it as well as they could, and without properly utilizing it, the rays / sec are maybe 25% of the standard nerfstudio dataset speeds (which are 10% of the size of the phototourism datasets). To do this, we specify a collate function in the existing datamanager configs, and we make a subclass of the vanilladatamanagerconfig named VariableResDataManagerConfig which will have a custom collate function storing the images in a list, and using better defaults for caching in the dataloader, caching ~40 images at a time. The slowest part of using a list is pixel sampling, where we added a custom if case for if our pixel sampler receives a batch of images that are in a list, and the lack of ability to vectorize operations over a list means that caching in the dataloader is even more important.
2022-11-25 22:16:11 -08:00
Nikhil Mohan 1a24f3e58c Refactor Data Pipeline (#896)
* add none check

* black formatting

* cleanup

* cleanup

* fix tests

* remove new line

* init

* toy example working

* semantic nerf working

* simplify pixel samplers

* cleanup + fix tests

* comments

* fix tests

* fix tests
2022-11-06 07:40:43 -08:00
Ethan Weber 05ba4b6b00 semantic nerfw (#640)
transients not working right now
2022-10-03 20:36:32 -07:00
evonneng 968959bdc9 quickstart updates + (other nits+links added to repo) (#633)
* small changes

* rewrite quickstart + readme

* fixed all licensing headers

* nits
2022-10-03 16:01:16 -07:00
Ethan Weber e13d401b16 restructure codebase pre release (#602)
* moving a lot

* building
2022-10-01 16:36:25 -07:00