Commit Graph
1648 Commits
Author SHA1 Message Date
Ethan Weber 7dfca12cfd pair pixel sampling with a mask 2023-08-04 15:17:28 +00:00
Ethan Weber 96abe705e0 changes 2023-08-04 14:28:31 +00:00
Frederik WarburgandEthan Weber 9b03299d40 Train and eval data splits (Nerfbusters) (#2207)
* nerfbusters dataparser

* nerfbuster dataparser and visualization of eval views

* resolved some errors from  pyright

* enable ns-process-data to take two videos as input and use one for eval and the other for train

* resolve some pyright issues

* instead of data being a tuple, I added eval_data as another argument

* implemented train eval split for all cases

* implemented train eval split for all cases

* reverting equi changes

* reverting reality capture changes

* raise valueerror

* revert comment

* added split override code back to dataparser

* reverted one change

* using image border now

* adding docs

---------

Co-authored-by: Ethan Weber <eweb0124@gmail.com>
2023-08-04 10:11:55 -04:00
machenmusik 388fcb09e1 convert raw to JPG not TIFF, to avoid fast-downscale ffmpeg error (#2295) 2023-08-02 09:44:35 -07:00
machenmusik 5b4abc41e6 Allow same dimensions optimizations to be disabled (#2292)
Use ns-process-data images --no-same-dimensions ... to handle images sets with different dimensions properly.  (Fast downscaling assumes image dimensions are the same.)
2023-08-01 23:35:27 -07:00
71c6b9cf3d Monodepth Regularization (#2284)
* monodepth implemented

* device

* pair pixel sampler

* docstring

* removing unused imports

* fix bugs

* cache in cache dir, no longer edit json file

* moving to depth nerfacto

* fixed bug using median depth by accident

* removing depthdataset from nerfacto

* added pixelsamplerconfig

* added equirectangular warning back

* bug fix

* delete sample_config.py + small fixes

* rename to pixel_sampler

* cleaning

* test

* passing num_rays_per_batch properly

* fixed transforms typo

* renaming in nerfacto

* remove depth files from test data

* removed defaulting to base pixel sampler with mask

* adding patch_size back

---------

Co-authored-by: AdamRashid96 <abrashid@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-08-01 18:52:47 -07:00
Brent Yi 72da747fc2 Remove system-level CUDA install from install instructions (#2289)
* Remove system-level CUDA install from Linux install instructions

* Remove system-level CUDA from Windows install instructions

* Un-flip CUDA 11.7 / 11.8
2023-08-01 22:50:22 +00:00
Brent Yi dab39c25b4 More robust install instructions (#2277)
Add cuda-toolkit install to readme/docs
2023-08-01 20:35:22 +00:00
AX-I 98d36ed7d9 Option to process images on GPU for speed at the expense of memory (#2288)
* Fix pixel sampler cpu bottleneck

cpu usage reduced 1700% -> 100%, and even more if using multi-gpu training

* Convert to config option

* Change optional to bool
2023-08-01 11:12:30 -07:00
Sasha Sax 7e8d67f8a8 Add depth_unit_scale_factor to sdfstudio_dataparser (#2285)
Depth supervision for `depth-nerfacto` was added in #1173. This patch extends that support to `sdfstudio` datasets ([as originally intended](https://github.com/nerfstudio-project/nerfstudio/issues/1173#issuecomment-1494633338_)) by adding `depth_unit_scale_factor` to `sdfstudio` metadata.
2023-08-01 09:03:01 -07:00
Fernando Rivas-Manzanequeandjperezh e6cd95109b Moved Volinga as an external method (#2282)
* Moved Volinga as an external method

* Update external_methods.py for black

* Formatted with black

---------

Co-authored-by: jperezh <jperezh@arquimea.com>
2023-07-31 07:34:38 -07:00
Matthew Tancik ce6b9bdd5e Update index.md (#2279) 2023-07-28 11:18:07 -07:00
Ethan Weber da813549e8 ns-render with depth colormap support (#2125)
* can render depth with a colormap

* include colormap_options as argument
2023-07-27 14:06:54 -04:00
hturki 4d80f0ceee Use correct pixel area in volumetric sampler (#2268) 2023-07-25 18:05:40 -07:00
Frederik Warburg ad73c5142e nerfbusters doc (#2208)
* nerfbusters doc

* ordered alphabetically and added nerfbusters to list of 3rd party methods
2023-07-25 22:52:59 +02:00
Cyrus Vachha 8682e481b1 Render VR180 video (#2261)
* Render VR180 video

Adding the VR180 camera model to render VR180 video and images. I condensed the omnidirectional stereo and VR180 render post processing in the render script. I have updated the documentation for VR180 support.

* Condensing formatting of camera types

Condensing the if statement for camera type intrinsics.
2023-07-24 07:26:20 -07:00
hturki 4f2157cd58 support packed samples in semantic renderer (#2247) 2023-07-20 09:48:25 -07:00
hyperzy e7fd1aefa2 Load schedulers from ckpt in a backward-compatible manner (#2236) 2023-07-18 18:02:15 +00:00
Nicholas PfaffandJonas Kulhanek 9f487a464f Implement nerfacto alpha transparency training (#2165)
* Implement nerfacto alpha transparency training

* Stop adding all ones alpha channel and clean up blend_background

* Make background_color a required argument in blend_background

* Enable alpha transparent training by default

* Improve background_color typing

* Blend uniform background color for metric computation

* Fix typing error

* Clean up typing

* Fix background_color to outputs assignment

* Model outputs returns unblended RGB (black bg) for random bg

* Update renderers

* Fix comments

* Add support for transparent BG for other methods

* Minor refac

* Fix get_background_color

* Fix typing error for blend_background

* Fix comments

* Export utils support transparency

* Pylint fix

* Fix pcd export masking

* Filter pcd view directions based on opacity

* Clean up get_rgba_image

---------

Co-authored-by: Jonas Kulhanek <jonas.kulhanek@live.com>
2023-07-18 12:54:52 +02:00
Khoa Nguyen-Tuanandkhoa dbd4092dbb Add std for eval (#2239)
* Add std for eval

* change default value of get_std

---------

Co-authored-by: khoa <tuankhoanguyen94@gmail.com>
2023-07-17 15:02:32 -07:00
Justin Kerr e417ee07e7 Minor kwarg fix to be future-compatible with viser version (#2235)
* add kwarg to step to avoid breaking in newer version of viser
2023-07-16 18:27:36 -07:00
Yueh-Cheng Liu 劉岳承andMatthew Tancik 8cd9beccd5 Fix arguments that are not passed from config to NerfactoField (#2186)
* Fix arguments that are not passed from config to NerfactoField

* Add comments

* Reset base_res and features_per_level config of nerfacto-big and nerfacto-huge to default to match the original behavior

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-07-15 14:24:37 +00:00
Sebastiaan 6329f53a5c Remove superfluous -r1 (fix ffmpeg/colmap video) (#2231) 2023-07-14 10:32:58 -07:00
Matthew Tancik 54d26814d9 Update torchmetrics (#2223) 2023-07-13 18:02:19 -07:00
Saurav Maheshkar 0cb98fbdd8 refactor: get_normalized_directions (#2216)
* refactor: normalized_directions

* fix: directions shape
2023-07-14 05:44:07 +05:30
machenmusikandSebastiaan Meijer 91291bc52c Fast downscaling for video and image processing (PR2168 plus images) (#2213)
* Merge video image extraction and downscaling

* Linting

* Fix extracting all frames, only apply vfr when thumnailing

* integrate downscale chains into image processing

* fast polycam upscale/downscale depth

* black

* black

* Update process_data_utils.py

Remove code change from other branch.

* use same suffix for downscale, don't force png

* black

---------

Co-authored-by: Sebastiaan Meijer <seb@stiaan.io>
2023-07-13 18:54:19 -04:00
Zhizheng XiangandMatthew Tancik edbe319924 Add documatations for TensoRF (#2209)
* add tensorf intro video

* add tensorf images

* update images to adapt dark mode

* add documentation for TensoRF

* grammer check

* add some anotations

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-07-13 22:43:11 +00:00
Jeff CookandMatthew Tancik 365e4c6663 Try non-strict model loading on RuntimeError. ... (#2214)
* Try non-strict model loading on RuntimeError. ...

This allows tcnn models that contained a torch hash_table to load.
In particular, it allows us to load tcnn models prior to PR #2175.

cf. https://github.com/nerfstudio-project/nerfstudio/pull/2175

* Only allow non-strict loading when called with defaults.

* Reflow to avoid code duplication; blacken.

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-07-13 22:22:36 +00:00
Victor Rong 7acaed11a0 Reorient point cloud normals for better surface reconstruction (#2220)
Implemented normal reorientation
2023-07-13 15:01:53 -07:00
Cyrus Vachha 554dec77f2 Omnidirectional Stereo Update - Alignment and Compositing (#2217)
Update to omnidirectional stereo rendering (VR 360) to fix slight positional alignment to accurately composite with Blender objects. Modified documentation for ODS and the Blender add-on to include details on Blender camera settings to composite with the Nerfstudio ODS camera model.
2023-07-13 07:59:10 -07:00
hyperzy cebe787bed Add saving/loading scheduler state_dict (#2203)
- Add saving/loading scheduler stat_dict
2023-07-12 14:14:59 +00:00
machenmusik 49a4be635d Fix colmap vocab_tree_path (#2195) to work with Windows (#2204)
* Fix colmap vocab_tree_path to work with Windows

* black
2023-07-12 07:03:23 -07:00
Riley 6cb10ae682 Adding input handling for RFFEncoding's get_out_dim(self) (#2201)
fixed RFFEncoding's get_out_dim(self) to incude dims of input if include_input
2023-07-11 22:56:12 +00:00
Paul-Edouard Sarlin 70adb01b4c Pin versions of pyceres+pixsfm, up pycolmap (#2199)
* Pin versions of pyceres+pixsfm, up pycolmap

* Update comment
2023-07-11 17:25:07 +00:00
hyperzy e9a5a66a54 Default fied_heads with instances will cause shared parameters (#2197)
* - Bug report. Modified the default parameters of NeRFField to avoid shared RGBFieldHead

* - Modified to ensure minimal changes

* - Modified type checking
2023-07-11 12:02:48 -04:00
Jan-Niklas Dihlmann 904e1352d9 Fixing Colmap VocabTreeMatching command to support spaces in Filename (#2195)
fix: colmap vocab tree file path error
2023-07-11 07:51:03 -07:00
machenmusik 82b3f70f87 Avoid zero duration crash (#2193)
Avoid zero duration divide-by-zero crash when processing too quickly.
2023-07-10 22:09:44 +00:00
b86a4251a9 Feature/run_beta_viewer (#2157)
* add `build-arg` cuda version to dockerfile

* [Feature] add 'vis' option to 'ns-viewer' script

* fix pylint

* [Fix] fix data format

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-07-07 22:01:14 +00:00
Georg HessandMatthew Tancik 492fa0e67b Remove pytorch hash table if tcnn implementation is used (#2175)
* fixes div by 1 error

* removes pytorch hash_table if tcnn is used

* formatting

* test fixes

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-07-06 17:55:22 +00:00
Weibo GaoandWeb 77cf4c88ed fix the ZeroDivisionError occured in int(image_width / aspect_ratio) when aspect_ratio equals zero (#2177)
fix the ZeroDivisionError occured in int(image_width / aspect_ratio)

Co-authored-by: Web <iamwebgao@163.com>
2023-07-06 10:38:10 -07:00
Weihang Li c7def8ce8c Fix image saving format issue in render (#2172)
* fix scannet dataparser pose issue

* camera pose corrected

* fix image format saving issue
2023-07-04 14:07:04 -07:00
KaerMorh 18ab50dc6b Correct path notation for Windows in documentation (#2171)
Fixed an error in the documentation where the incorrect path notation was used for Windows systems. The documentation had paths denoted with "./", which is incorrect for Windows. This has been corrected to ".\", the appropriate notation for Windows.

This change ensures that Windows users correctly follow the instructions in the documentation without encountering any path-related issues.
2023-07-04 08:14:15 -07:00
Daniel CrispellandBrent Yi 8ed2a2d667 Make all SDFFieldConfig params configurable via constructor (#2167)
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-07-03 21:00:38 +00:00
Matthew Tancik 061d691d43 Default to tcnn (#2159) 2023-07-02 21:36:28 -07:00
Brent Yi d6bb8bd8e5 Fix use_appearance_embedding for instant-ngp (#1999)
* Attempt fixes for instant-ngp

* Run `ns-dev-test`

* Remove NGP field implementation

* Test nerfacto field instead of ngp field

* Black
2023-07-01 18:54:27 +00:00
Ilya Basharov 5c4b0a1dd6 add build-arg cuda version to dockerfile (#2139) 2023-07-01 17:06:05 +00:00
Milan van Wouden f4be2b729a Fix typo in nerfacto.py (#2149) 2023-06-30 08:44:53 +00:00
hturkiandMatthew Tancik 0719026c15 Add option to render evaluated images (#2135)
* Add option to render evaluated images

* add args

* fix formatting

* attempt to fix pyright

* attempt 2

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-06-29 23:26:56 +00:00
Jonáš KulhánekandMatthew Tancik b3df46314c Remove breakpoint from colmap_dataparser (#2141)
Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-06-29 17:03:29 +00:00
Konstantin Kozlovtsev 000272f895 Fix unnecessary encodings calculation in tensorf field (#2145)
Fixed unnecessary encodings calculation in tensorf field
2023-06-29 09:54:59 -07:00