Commit Graph
58 Commits
Author SHA1 Message Date
David McAllisterandMatthew Tancik 4c2e200098 TensoRF go brrrrrrr (#1609)
* pdfsampler no longer includes uniform samples

* Lint

* Typo fix

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-03-15 20:27:37 -07:00
Pablo Vela ddfa463f4d NeuS Implementation (#1444)
* add initial method configs and data manager

* add neus scheduler

* add initial surface model and neus

* add Neus ray sampler

* add sdf field

* move scale/shift calc to math utils

* add debug and fix sdf_dataset typo

* update vscode neus debug

* fix dataset and get initial working version

* add magic number from original repo

* black fixes

* clean up get weights/transmittance from alphas

* clean up typos in sdf datamanager

* type annotations on neus scheduler and remove unused params

* clean up and annotate sdf field along w/ other fixes

* more clean up

* remove unnessary directions norm

* resolve comments

* update neus scheduler to follow refactor

* fix linter errors

* fix some typos

* remove get_weights_from_alpha

* more comment fixes

* more linter error fixes

* clean up and add comments

* more little fixes for types/comments

* more comments

* rename NeuS Scheduler to DelayedScheduler

* rename DelayedScheduler to CosineDecayScheduler
2023-03-12 16:01:18 -04:00
Georgy Ponimatkin f48fa6a923 Instant ngp args (#1549)
* Expand instant-ngp args.

* Fixed misplaced string.
2023-03-02 23:49:30 +01:00
Fernando Rivas-ManzanequeandMatthew Tancik 9053643932 Volinga model and NVOL documentation (#1486)
* added volinga model to NeRFStudio

* Removed unused dependencies

* Modified dockerfile to match main

* Corrected typo in core_code_checks.yml

* Corrected typo in core_code_checks.yml

* updated test with Volinga

* Added conversion to NVOL documentation

* [WIP] Refactor documentation and removing Volinga config

* Volinga use existing NerfactoModelConfig

* Added Unreal Engine documentation to extensions

* removed unused field class import

* Updated docs

* Documentation ready

* Update Dockerfile

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-02-25 14:49:53 -08:00
Ruilong Li(李瑞龙) 7dc7129cf8 fixes for nerf synthetic dataset (#1507)
* fixes for nerf synthetic dataset

* restore eval script

* fix format with black
2023-02-25 14:37:10 -08:00
Nikhil Mohanandmachenmusik a91b92f89d semantic_nerfw_configurability (#1467)
* fix semseg vis

* semantic configs

* change defaults

* Fix type

* format

* fix imports

* suggested fix for issue 1465

* fix docstring placement; avoid log spam and runtime errors

`nerfstudio\models\semantic_nerfw.py:81: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  self.colormap = torch.tensor(self.semantics.colors).to(self.device)`

* hide semantics output choice, since it kills the viewer

* revert some changes

* caps

* move both image and mask to GPU to avoid runtime error

as suggested in https://github.com/nerfstudio-project/nerfstudio/issues/1465#issuecomment-1441117562

---------

Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
2023-02-22 22:21:04 -05:00
Jake AustinandMatthew Tancik 50dc5d6cd7 Triplanes (#1456)
* Triplanes working

* Doing the TODOs, better variable names

---------

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-02-22 16:23:12 -08:00
Matthew Tancik f3a34bbf95 Refactor schedulers (#1466)
* Refactor schedulers

* lint

* lint
2023-02-21 12:51:51 -08:00
Sacha Lewin 151b1060fe Uniform Sampler in Nerfacto (#1464)
* Allow black/white background color in Nerfacto

* Use white background in Nerfacto Blender Benchmark

* Fixed unused import

* Updated index for license headers script

* Removed debug code in NGP models

* Uniform initial sampler config in Nerfacto

* Undo changes in settings.json

* Removed unwanted settings file

* Removed unneeded config parameter

* Use uniform sampler on Blender benchmark
2023-02-21 09:17:57 -08:00
Matthew Tancik ceda935e64 Fix depth nerfacto exporting (#1453) 2023-02-19 13:53:01 -08:00
Matthew Tancik 73f0f5a16a Fix mesh export normals (#1452)
* Fix mesh export normals

* lint
2023-02-19 13:43:43 -08:00
Sacha Lewin 8e4ae6bebf Allow use of black or white background in Nerfacto models (#1441)
* Allow black/white background color in Nerfacto

* Use white background in Nerfacto Blender Benchmark

* Fixed unused import

* Updated index for license headers script
2023-02-17 20:39:10 -08:00
NCJ 2878aff1c5 Fix LPIPS data range (#1425) 2023-02-14 10:18:33 -08:00
Yiming Zhou 8c9c2ba2ab Add support for sparse depth maps (#1373)
Add support for sparse depth maps (#2)

* sparse depth maps

* pr comments

* unified loss

* minor

* lint
2023-02-07 16:26:05 -08:00
Matthew Tancik 59d7e99c80 Create shaders (#1344) 2023-02-03 11:21:54 -08:00
Jeffrey Hu a0976d1ff8 Fix depth-supervised nerf when using z-buffer depth. (#1340)
* fix z buffer depth

* remove script
2023-02-02 18:22:55 -08:00
Matthew Tancik 2e79c93cd2 Fix normals visualization (#1341) 2023-02-02 18:06:11 -08:00
LSongandmachenmusik e73692e599 nerfplayer with ngp and nerfacto (#1277)
* nerfplayer with ngp and nerfacto

* update comments

* update cpp extension build

* avoid Windows repeated CUDA compile

---------

Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
2023-01-31 11:56:28 -08:00
Justin Kerr c8fdc1a7cb Increase max_res of first proposal network (#1282) 2023-01-24 13:23:37 -08:00
Terrance Wang 73bffe058f Fix tensorf chkpt loading (#1266)
* setting tensorf default from wandb to viewer

* adding functionality to adjust model depending on the step of the checkpoint that is loaded

* cleaning code

* bug fix

* fixing style
2023-01-20 12:00:51 -08:00
2120bd5dd9 CropBox GUI implementation (#1205)
* fixed an exception bug in case of num_samples is 0

fixed an exception bug when num samples is 0 , 
"IndexError tensors used as indices must be long, byte or bool tensors"

change type to long ( was float32)

* implemented crop box in gui. For now works on instant-ngp-bounded

* fixed import bugs

* update aabb every time and not only on training

* prevent crash when chosing accomulation and crop box

* Checkbox

* max_box is bounded by scevne box in the gui

* implemented altenative intesection to nerfacc

* convert str to bool correctly

* fixed isort bug

* fixed isort bug2

* update viewer version

* remove json bug

* Update UI

* Force black background

* Fix formatting

Co-authored-by: dror <you@example.com>
Co-authored-by: Matt Tancik <tancik@berkeley.edu>
Co-authored-by: Matthew Tancik <mattt4001@gmail.com>
2023-01-19 17:39:37 -08:00
JamesPerlman e575f128d7 Update incorrect comment (#1226) 2023-01-16 12:42:22 -08:00
matsurenandMatthew Tancik 408ab709ec Fix typo (#1231)
* Fix typo COLAMP->COLMAP

* Fix more typo

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2023-01-16 12:12:55 -08:00
Michal Pandy 3da877050d Depth supervision for Nerfacto (#1173)
* Implement depth supervision

* Fixes

* Change normalize dtype

* refactor

* Change depth weight

* Small refactor

* Change docstring

* Removu unused import

* Add assumption to depth comment

* Implement URF loss

* Remove default depth loss type param

* URF fixes

* Fix depth loss scale

* Visualize gt depths

* Fixes & documentation

* Documentation and formatting

* Fix formatting problems

* Fix linter problems

* Fix tests

* Fix tests
2023-01-02 10:13:38 +01:00
Ruilong Li(李瑞龙) 028df82be6 upgrade nerfacc to 0.3.x with CUB accel (#1187) 2022-12-31 07:28:11 +08:00
Brent Yi 5495a8e146 Type and docs fixes (#1170) 2022-12-26 09:04:24 -07:00
dozeri83anddror 96f269cc1a instant-ngp-bounded model (#1116)
* inerted a new model configuration instant-ngp-bounded

* added more remarks and small fixes

* went back to old near plane

* added the oprtion to get a file as well as directory in nerfstudio data parser

* pretify prev commit

* review fixes

* more clear explanation in README.md

* reverted readme changes

* fixed merge bug

* review changes. update config.data docs support for json files inputs

Co-authored-by: dror <you@example.com>
2022-12-13 13:05:48 -08:00
Matthew Tancik 676f816156 Bump max res default (#1106) 2022-12-11 13:03:35 -08:00
Zak Murezandzmurez a7df91edc8 config nerfacto (#1020)
Co-authored-by: zmurez <zmurez@eng.ucsd.edu>
2022-11-27 15:56:06 -08:00
b829499c12 Computing and predicting normals (#905)
* staring normals PR

* code runs but visualization looks bad

* Camera and world coordinate conventions (#889)

* documented coordinate conventions without a figure for now

* another sentence

* update frames docs

* explain intrinsics and extrinsics

* more comments

* fix build

* Start viewer with more accurate speed calculation (#899)

Start viewer with more accurate speed calc

* Minor doc updates (#903)

* fix conflicts

* code runs but visualization looks bad

* normals implemented and vis fixed. still debugging quality

* minor name change

* comment

* fix black

* lint

* predicting normals

* adding detach

* predicted normals

* improved normals prediction performance

* cleaning code

* cleaning code

* cleaning code

* adding type to function param

* fixing tensorf forward func

* reverting tensorf changes

* fix style with tensorf field

* adding normals check to tensorffield

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
Co-authored-by: terrance <terrance.3512@gmail.com>
2022-11-20 13:22:37 -08:00
Matthew Tancik a7b2af185c Cleanup configs (#963) 2022-11-15 16:56:56 -08:00
Julian KnodtandMatthew Tancik 9ff167d705 Add temporal distortion and D-NeRF model (#825)
* Add D-NeRF temporal distortion and D-NeRF model

This allows for temporal distortion to be enabled on a variety of models through config
parameters, without an explicit model. Temporal distortion and MIP-NeRF are not currently
jointly runnable but can be changed in the future.

* Simplifying changes

* Simplifying changes

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2022-11-15 14:29:29 -08:00
Nikhil Mohan 1a24f3e58c Refactor Data Pipeline (#896)
* add none check

* black formatting

* cleanup

* cleanup

* fix tests

* remove new line

* init

* toy example working

* semantic nerf working

* simplify pixel samplers

* cleanup + fix tests

* comments

* fix tests

* fix tests
2022-11-06 07:40:43 -08:00
Terrance WangandMatthew Tancik 7106f42fcc tensorf (#586)
* tensorf wip

* model changes

* tuning model

* changing imports

* merging

* changing rff encoding to nerfencoding, basis matrix should be properly updated, changed density activation, lr scheduler

* debugging tensorf

* debugging tensorf

* tensorf debugging

* tensorf debugging

* tensorf debugging

* tensorf debug

* debugging tensorf, fixed bug with collider

* tuning model, ray sampling

* adding thresholding for ray computations

* removing unused imports

* making more params configurable in config, adding spherical harmonics

* removing scene contraction, adding arg docstrings

* fixing style

* Fixing bug

Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
2022-11-02 14:32:03 -07:00
Matthew Tancik ca69adaaec Add median depth calculation (#877)
* Add median depth calculation

* lint

* Fix test
2022-11-01 15:05:16 -07:00
dolhaszandadolhasz def6ba5c51 Fix semantic NeRF config bug (#838)
Fix semantic nerf config bug

Co-authored-by: adolhasz <adolhasz@matterport.com>
2022-10-27 08:44:34 +03:00
Matthew Tancik 9bed69601f Support python 3.7 (#802)
* Support python 3.7

* Update docs
2022-10-18 16:49:27 -07:00
Justin Kerr a4755a1349 Sample proposal networks on a schedule to speed training (#770) 2022-10-13 20:14:21 -07:00
Matthew Tancik 5f2739e634 Fix tb and wandb crash (#769) 2022-10-12 19:28:58 -07:00
Matthew Tancik d24d82d261 Reduce memory usage of nerfacto model (#766)
* Reduce memory usage of nerfacto model

* Tweak config
2022-10-12 18:01:58 -07:00
2150bad0e4 Reducing flicker artifacts for nerfacto method (#750)
* pose opt default nerfacto.

* added SO3 cam pose optimization which is slightly faster (~2.2m/iter) than BARF

* added occupancy grid with placeholder until nerfacc is updated to query density from it

* added occupancy grid sampling as the level 0 in proposal hierarchy, relies on nerfacc 0.1.7 (on pypi)

* merge main

* merge main

* minor changes to sampling

* adjusted sample number

* refactor occ grid stuff, settle on decent params

* nograd proposal sampling

* param tweaks

* varying resolution proposal networks

* params

* merged main

* optimizer update

* remove occ grid

* remove double nerfacto samples

* set single jitter to true

* remove pdb

* more changes

* fix method defaults

* fix pylint

* remove occupancy fn

Co-authored-by: akristoffersen <akristoffersen@berkeley.edu>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2022-10-11 14:21:00 -07:00
Ruilong Li(李瑞龙) e94b093b57 bump nerfacc to 0.2.0. bug fix for depth vis. (#744)
* bump nerfacc to 0.2.0. bug fix for depth vis.

* skip emtpy region for nerfacc raymarching
2022-10-09 19:16:30 -07:00
Ethan Weber 186d590105 remove print (#679) 2022-10-04 23:57:59 -07:00
Ethan Weber b8f85fb603 doubling nerfacto samples on eval (#675)
doubling samples and eval num ray options
2022-10-04 22:57:24 -07:00
Matthew Tancik 21f58252c1 remove tensorf model (#660)
* remove tensorf model

* lint
2022-10-04 16:39:47 -07:00
Matthew Tancik 654f070553 Remove mipnerf 360 (#659)
remove mipnerf 360
2022-10-04 16:29:11 -07:00
Matthew Tancik 6162f9d81f Tancik/remove valid mask (#655)
* Install updates

* Remove valid mask logic
2022-10-04 13:54:39 -07:00
Matthew Tancik 4d5a1363b1 Update docs (#649) 2022-10-04 09:23:31 -07:00
Ethan Weber 05ba4b6b00 semantic nerfw (#640)
transients not working right now
2022-10-03 20:36:32 -07:00
Matthew Tancik 8093a359e8 Tune proposal network (#638) 2022-10-03 18:38:36 -07:00