Commit Graph
600 Commits
Author SHA1 Message Date
Chung Min KimandJustin Kerr 4cdeaa6eec Track all viser GUI folders to avoid folder duplicates (viewer-beta) (#2466)
* Track all viser GUI folders to avoid folder duplicates

* lint

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2023-09-25 13:14:51 -07:00
Jonáš Kulhánek 2864c52618 Fix COLMAP dataparser masks and depthmaps (#2467) 2023-09-25 08:37:08 -07:00
Maxim Bonnaerens 53037ba8a7 Allow custom sampler in ProposalNetworkSampler (#2464)
* Allow custom sampler in ProposalNetworkSampler

This allows custom (pdf)samplers in distinct repositories while still
using the base ProposalNetworkSampler.

* sampler should inherit from pdf sampler

* docstring
2023-09-24 12:06:16 +02:00
Maxim BonnaerensandMatthew Tancik 52f26e82b3 Do not materialize mlp_head if no field_heads (#2457)
In case it is used in a proposal network we avoid initializing the head.

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-09-22 15:36:51 +00:00
フラハティ 仁 2fd8b59d20 Add background_color config to keep consistent with other models (#2455)
* add backgorund_color config

* removed unused color import
2023-09-22 07:40:20 -07:00
Jonáš Kulhánek 76f8bcf953 Disable loading colmap points by default (#2451) 2023-09-21 13:51:35 -07:00
Justin Kerr e0d8f04267 Render oriented crops in viewer beta (#2447)
* add obb to the rendering pipeline

* useless line

* lint

* remove todo

* pyright

* more pyright
2023-09-19 16:48:54 -07:00
Brent YiandJustin Kerr 3e1d4af821 Add share link support for beta viewer (#2445)
* Add share link support for beta viewer

* More consistent CLI experience

* fix viewer jitter by adding atomic update on camera pose

* change to make_share_url

* fix

* remove incorrect assert

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2023-09-19 16:22:55 -07:00
フラハティ 仁 b0e700137b Allow the option for timestamp to be generated for ns-train --load-config (#2442)
* Fix timestamp for ns-train --load-config

* Remove whitespace
2023-09-18 21:13:48 -07:00
Jonáš Kulhánek ef2fd3dbe1 Fix import error when tcnn installed but CUDA runtime not available (#2429) 2023-09-15 17:18:23 -04:00
Sid Mehta 7269a7ee16 Add Comet Logging (#2431)
* initial commit

* fixed naming

* added comet_ml as depedency

* added comet to docs

* Update custom_gui.md

* Update models.md

* Update benchmarking.md

* added Comet to ReadMe

* added decorator

* fixed link and added comet examples

* fixed link and added comet example

* fixed format

* fixed type ignore error

* fixed readme typo

* fixed typo
2023-09-15 09:21:09 -07:00
anc2001andBrent Yi 805e6e702d fix pixel sampling with masks issue when data is list (#2369)
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-09-14 16:11:15 +00:00
Jonáš Kulhánek e9330d515d Colmap parser: fix downscale for complex paths (#2425) 2023-09-13 10:55:24 -07:00
Maxim BonnaerensandBrent Yi cc98fb6bd4 Fix benchmarking: apply get_background_color in renderer and set profiler to none (#2397)
* apply get_background_color and set profiler to none

* Apply override first

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-09-11 18:07:52 +00:00
Jonáš Kulhánek 570ccf3c0a Rename COLMAP 3D point features (#2417) 2023-09-11 09:00:44 -07:00
Jonáš Kulhánek ead9b37c65 Allow colmap parser to load 3D points (#2408) 2023-09-11 07:50:18 +00:00
Sebastiaan 77ac7d0cf8 Fix issue when extracting all frames from a video. (#2413)
* Remove superfluous num_frames

* process_data video: Always use -vsync vfr, even when extracting all images.
2023-09-08 16:00:10 -07:00
drewgilliam 06e59ed1ec camera/plugin docs (#2412)
* api docs for cameras & plugins

* fix docstrings
2023-09-08 09:14:16 -07:00
Jonáš Kulhánek 6af0bcfbf8 Unify dataparser and method discovery (#2398)
* Unify external dataparser registration

* Unify dataparser and method discovery
2023-09-08 08:26:41 +00:00
85ef567682 Flesh out viewer beta (#2086)
* Increase max_res of first proposal network

* fix render statemachine logic for viewer beta

* clicking on cameras in the viewer

* fix fov calculation, update camera positions while training

* cleanup

* backup

* fix configure theme

* Fix theme configure for latest dev version of viser

* Fix runtime errors

* add set up direction button

* add pause resume button

* Export panel

* Render tab skeleton

* make responsiveness more snappy by stopping training

* depth composite

* start implementing viewercontrol in new viewer

* refactor background img to use depth and add depth layering option

* show/hide images

* speed up rendering

* fix crop color background

* crop

* rotated crops

* rotated crops

* transform handle on crop box

* fix xyzw for crop

* oriented crop export for pointclouds, not for meshes yet

* Initial render panel

* Fix export button

* Theme tweaks

* fix background issue in cropping

* export rotated poisson meshe

* double normals definition in export panel

* Visual nits

* remove tsdf from export panel

* Add licenses

* Fix some CI errors

* Type fixes

* Fix tests

* Fix numpy broken

* various bugfix

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: terrancewang <terrance.3512@gmail.com>
2023-09-08 05:10:00 +00:00
Zhensheng YuanandMatthew Tancik 9625ab713c Fix the problem that camera will be freezed when moving forward (#2404)
* fix camera freezed when moving forward

* simplify code

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-09-08 00:31:02 +00:00
drewgilliam 4d0033343c NERFSTUDIO_METHOD_CONFIGS callable MethodSpecification (#2385)
* NERFSTUDIO_METHOD_CONFIGS allow method specification as callabale (function or class)

* docs - NERFSTUDIO_METHOD_CONFIGS accepts function or derived class
2023-09-05 13:55:08 -07:00
Maxim Bonnaerens 242c23f0f0 cast background to device (#2381)
* cast background_color to device

* make onliner instead
2023-08-29 15:07:43 -07:00
Justin Kerr d56338bce1 Fixes crop background color override (#2373)
fix crop color background
2023-08-27 11:50:57 -07:00
elenacliuandAdamRashid96 ae5d569376 Fix: typo in PairPixelSamplerConfig's description (#2372)
Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
2023-08-26 03:09:44 -07:00
6e37b2b542 Enable SparseNerf depth ranking loss to be used with GT/Sensor depth (#2312)
* add ranking loss to depth-nerfacto with GT depth

* formating

* pair pixel sampler bug fix from pr #2315

---------

Co-authored-by: Sasha Sax <asax@stanford.edu>
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: AdamRashid96 <abrashid@berkeley.edu>
Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
2023-08-26 02:56:52 -07:00
Zhan XuandMatthew Tancik ebceb52f55 save sfm depth as png to keep 16bit. save depth_file_path as relative… (#2317)
save sfm depth as png to keep 16bit. save depth_file_path as relative path

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-08-24 22:30:26 +00:00
Rohan Mathur 48135ee4c8 Added a pytorch implementation of OmniCV's to remove equilib package usage (#2350)
* added pytorch omnicv implementation in order to not use GPL equi2pers

* renamed rotation matrix function to decrease confusion

* fixed docstrings

* fixed linting

* removed pyequilib from dependencies
2023-08-20 21:34:18 -07:00
Ethan Weber 9315cad5f5 Switching Nerfstudio Dataset data to GDrive URLs (#2358)
* switch urls to gdrive

* added comment with explicit link to gdrive url
2023-08-19 13:48:28 -07:00
James Gardner cba8d22d3b Optimizers.scheduler_step() bug fix (#2352)
optimizer bug fix
2023-08-18 09:01:50 -07:00
Sasha SaxandMatthew Tancik 9cd957f776 Update pixel_samplers.py (#2327)
* Update pixel_samplers.py

Now uses the specified batch_size param in sample_method()

* black

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2023-08-17 18:04:51 +00:00
Brent Yi 6d51af1a24 viser 0.1.0, fix type checker errors (#2343)
* Pin to viser 0.1.0, fix type checker errors

* gui_folder => add_gui_folder
2023-08-15 23:15:35 +00:00
Justin Kerr 5de83ac50a Circle crop to images processing pipeline (#2341)
add circle crop to images data processing pipeline
2023-08-15 14:32:44 +09:00
elenacliu bc8341378a fix #2318: add train/test/eval split for instant ngp dataparser (#2335)
* fix #2318: add train/test/eval split for instant ngp dataparser

* fix: instant_ngp_dataparser.py format
2023-08-14 04:24:34 +00:00
Haoyuan Wang 3455fb749f fix: ColmapDataParser downscaling images error. (#2337) 2023-08-13 21:16:47 -07:00
Matias Turkulainen 4fe8e657d2 Rename num_cameras to camera_indices (#2326)
rename num_cameras -> camera_indices
2023-08-10 15:36:01 -07:00
machenmusik cb2405d3ea When generating depth images, use torch_compile with ZoeDepth for compatibility (#2303)
* Use torch_compile with Zoe for compatibility

torch_compile module support

Previously the compatibility utility torch.compile worked for functions, but not for modules as torch.compile was originally employed here.

* torch_compile -> Any
2023-08-09 17:08:17 -07:00
Ethan Weberandmachenmusik 9b6010ea0d Adding support for PairPixelSampler and PatchPixelSampler masks (#2305)
* changes

* pair pixel sampling with a mask

* reverting changes

* revert changes

* remove line in erode

* support for patch pixel samplerr masks

* fix

---------

Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
2023-08-05 10:40:23 -04:00
7e90cb7989 PixelSampler Fixes (#2298)
* monodepth implemented

* device

* pair pixel sampler

* docstring

* removing unused imports

* fix bugs

* cache in cache dir, no longer edit json file

* moving to depth nerfacto

* fixed bug using median depth by accident

* removing depthdataset from nerfacto

* added pixelsamplerconfig

* added equirectangular warning back

* bug fix

* delete sample_config.py + small fixes

* rename to pixel_sampler

* cleaning

* test

* passing num_rays_per_batch properly

* fixed transforms typo

* renaming in nerfacto

* remove depth files from test data

* removed defaulting to base pixel sampler with mask

* adding patch_size back

* Warning for notimplemented error and patch_size fix

* fix config.pixel_sampler check

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: Ethan Weber <ethanweber@berkeley.edu>
Co-authored-by: Ethan Weber <eweb0124@gmail.com>
2023-08-04 10:27:35 -04:00
Frederik WarburgandEthan Weber 9b03299d40 Train and eval data splits (Nerfbusters) (#2207)
* nerfbusters dataparser

* nerfbuster dataparser and visualization of eval views

* resolved some errors from  pyright

* enable ns-process-data to take two videos as input and use one for eval and the other for train

* resolve some pyright issues

* instead of data being a tuple, I added eval_data as another argument

* implemented train eval split for all cases

* implemented train eval split for all cases

* reverting equi changes

* reverting reality capture changes

* raise valueerror

* revert comment

* added split override code back to dataparser

* reverted one change

* using image border now

* adding docs

---------

Co-authored-by: Ethan Weber <eweb0124@gmail.com>
2023-08-04 10:11:55 -04:00
machenmusik 388fcb09e1 convert raw to JPG not TIFF, to avoid fast-downscale ffmpeg error (#2295) 2023-08-02 09:44:35 -07:00
machenmusik 5b4abc41e6 Allow same dimensions optimizations to be disabled (#2292)
Use ns-process-data images --no-same-dimensions ... to handle images sets with different dimensions properly.  (Fast downscaling assumes image dimensions are the same.)
2023-08-01 23:35:27 -07:00
71c6b9cf3d Monodepth Regularization (#2284)
* monodepth implemented

* device

* pair pixel sampler

* docstring

* removing unused imports

* fix bugs

* cache in cache dir, no longer edit json file

* moving to depth nerfacto

* fixed bug using median depth by accident

* removing depthdataset from nerfacto

* added pixelsamplerconfig

* added equirectangular warning back

* bug fix

* delete sample_config.py + small fixes

* rename to pixel_sampler

* cleaning

* test

* passing num_rays_per_batch properly

* fixed transforms typo

* renaming in nerfacto

* remove depth files from test data

* removed defaulting to base pixel sampler with mask

* adding patch_size back

---------

Co-authored-by: AdamRashid96 <abrashid@berkeley.edu>
Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-08-01 18:52:47 -07:00
AX-I 98d36ed7d9 Option to process images on GPU for speed at the expense of memory (#2288)
* Fix pixel sampler cpu bottleneck

cpu usage reduced 1700% -> 100%, and even more if using multi-gpu training

* Convert to config option

* Change optional to bool
2023-08-01 11:12:30 -07:00
Sasha Sax 7e8d67f8a8 Add depth_unit_scale_factor to sdfstudio_dataparser (#2285)
Depth supervision for `depth-nerfacto` was added in #1173. This patch extends that support to `sdfstudio` datasets ([as originally intended](https://github.com/nerfstudio-project/nerfstudio/issues/1173#issuecomment-1494633338_)) by adding `depth_unit_scale_factor` to `sdfstudio` metadata.
2023-08-01 09:03:01 -07:00
Fernando Rivas-Manzanequeandjperezh e6cd95109b Moved Volinga as an external method (#2282)
* Moved Volinga as an external method

* Update external_methods.py for black

* Formatted with black

---------

Co-authored-by: jperezh <jperezh@arquimea.com>
2023-07-31 07:34:38 -07:00
Ethan Weber da813549e8 ns-render with depth colormap support (#2125)
* can render depth with a colormap

* include colormap_options as argument
2023-07-27 14:06:54 -04:00
hturki 4d80f0ceee Use correct pixel area in volumetric sampler (#2268) 2023-07-25 18:05:40 -07:00
Cyrus Vachha 8682e481b1 Render VR180 video (#2261)
* Render VR180 video

Adding the VR180 camera model to render VR180 video and images. I condensed the omnidirectional stereo and VR180 render post processing in the render script. I have updated the documentation for VR180 support.

* Condensing formatting of camera types

Condensing the if statement for camera type intrinsics.
2023-07-24 07:26:20 -07:00
hturki 4f2157cd58 support packed samples in semantic renderer (#2247) 2023-07-20 09:48:25 -07:00