Commit Graph
1807 Commits
Author SHA1 Message Date
Cyrus Vachha 15e81d37dc Update documentation for 3DGS (#2780)
Adding 3rd party viewers for splats and other edits to gs documentation.
2024-01-18 09:51:15 -08:00
Brent Yi 368c9ecd12 Bump viser (#2779) 2024-01-17 15:35:08 -08:00
Justin Kerr 7d0ab6aca8 Docs for gaussian splatting (#2776)
* fill in doc page for GS
2024-01-17 19:44:29 +00:00
Matias Turkulainen a6a3dedc0e camtoworld is already a tensor in sdfstudio_data (#2777)
camtoworld is already a tensor
2024-01-17 11:22:34 -08:00
Brent Yi 05d305467f Reduce startup overhead (#2763)
* Reduce import overhead

* Bump tyro, suppress deprecated config values

* Bump viser

* Fix pyright
2024-01-17 01:42:10 +00:00
Too Yuen MinandBrent Yi e00da7de31 Update dependencies for aarch64 (#2750)
remove pymeshlab for aarch64

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2024-01-17 00:39:35 +00:00
Brent YiandJustin Kerr 609dc3ea5e Remove unused make_share_url arg in ns-viewer (#2749)
* Remove unused make_share_url arg in ns-viewer

* Update docstring

* Ruff

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-17 00:08:18 +00:00
Matthew Hoffman 32939e0d41 Remove deprecated dependency_links config (#2753)
* Move dependency_links to setup.cfg

[options] is not a valid table in pyproject.toml, but it is in setup.cfg: https://setuptools.pypa.io/en/latest/deprecated/dependency_links.html

* Remove setup.cfg

https://github.com/nerfstudio-project/nerfstudio/pull/2753#issuecomment-1893217202
2024-01-16 01:29:07 -08:00
Cyrus VachhaandBrent Yi 3e2271a656 Setting save_world_frame as False by default and Splat export panel in viewer (#2700)
* Setting save_world_frame as False by default

Setting save_world_frame as False by default for the exporter as well as in the viewer, since it is exports the point cloud in a different coordinate system from the Blender add-on and there is not a similar checkbox in the original viewer, so by default the exported point cloud would not be compatible. This flag can still be set true in the cmd command and in the new viewer

Also updating the blender docs to mention support for gs scenes

* Adding splat export panel in viewer

Adding the splat option in the export panel which returns a basic splat export cmd. Also renaming the splat exports as splat.ply instead of point_cloud.ply

* Showing export panels based on method

If gaussian splatting is running, then the crop box, mesh, and point cloud exports are hidden/disabled. Else, all but splat export is visible.

* Fix pyright type check

* Changing viewer model argument for export panels

* Update export_panel.py

* Update export_panel.py

* Update export_panel.py

* Update exporter.py

* Update exporter.py

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2024-01-15 22:04:22 +00:00
Matthew Hoffman f0aadf9de8 Use ruff as linter and formatter (#2765)
* Switch from black to ruff format

* Sort dependencies with ruff check

* Run ruff format
2024-01-15 13:43:14 -08:00
Brent Yi 69ed6d4093 Improve viewer banner when server is bound to 0.0.0.0 (#2761)
* Improve viewer banner when server is bound to 0.0.0.0

* Add comment

* Update viewer.py
2024-01-13 20:31:39 +00:00
SiN 44dafa3a4e Add --no-cache-dir option to pip install in Dockerfile (#2739) 2024-01-13 06:24:27 +00:00
Matthew Hoffman 114c5f7aa5 Fix apple silicon install 3.10+ (#2752)
* Fix install on darwin arm64 Python 3.10+

* Fix darwin arm64 3.11+ install
2024-01-12 12:59:04 -08:00
Brent Yi 55b7a2c8ef Update docs to recommend torch 2.1 (#2748) 2024-01-11 15:12:41 -08:00
Brent Yi 3b6ca91dbc Viewer depth compositing for Gaussian Splatting models (#2744)
* Viewer depth compositing for Gaussian Splatting models

* Tune
2024-01-11 04:04:00 +00:00
Matthew Hoffman e7e713e900 Fix Python 3.11 dataclass field default_factory defaults (#2746) 2024-01-10 17:12:54 -08:00
3ec10f8ea7 Undistortion should not occur for already-undistorted images (#2726)
* undistortion should not occur if pinhole (smoothes the images), and accordingly fix rounding of downscaled image size

* rounding downscaling, more consistent with other resizing methods in public datasets

* reverting to cpu image caching

* re-allow camera unimplemented

* compatibility with mask/depth

* reverted scale factor rounding, corrected mask undistort condition

* moved distortion_params update inside PERSPECTIVE type if-clause

* bumping opencv to 4.8, to deal with undistortion bug with distortion parameters of zero

---------

Co-authored-by: liorf <liorf@amazon.com>
Co-authored-by: alshan <alshan@amazon.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-10 12:44:57 -08:00
hturki 4e8c65cfbf Add Mill 19 dataset (#2742)
* add Mill 19 dataset

* Address PR feedback
2024-01-10 16:08:12 +00:00
hturki 7edd768240 Add PyNeRF external method (#2734)
* add pynerf method

* fixes

* fixes

* format

* rename k-planes references
2024-01-10 00:58:02 +00:00
Matthew TancikandJustin Kerr 451a70c1f3 Migrate to viser viewer (#2515)
* fix jittering in markdown in viewer beta

* Revert "fix jittering in markdown in viewer beta"

This reverts commit 70ade42e46.

* Migrate to viser viewer

* Fix docs

* print correctly formatted url in banner for viewer beta

* add deprecation warnings to viewer_elements.py

* fix export string scientific notation bug, change .5 to 2.0 sec transition time

---------

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-08 11:40:02 -08:00
Alexander KristoffersenandJustin Kerr ca9e6c1294 Fixed nerfstudio logo at the top of Colab notebook (#2733)
Fixed logo at the top of notebook

Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-08 18:39:41 +00:00
581ba95eab fixed not cache images for full image datamanager (#2692)
* rm no-cache option

---------

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
Co-authored-by: maturk <matiasturkulainen@gmail.com>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-08 10:19:38 -08:00
J.YandMatthew Tancik 638180c90a Improvement to Gaussian Splatting workflow (#2709)
* Necessary changes to establish baseline as reported by papers

- Port multiple train/eval split strategies from nerfstudio parser for colmap parser

- A number of fixes to establish solid baseline

- Avoid empty frames where no gaussians are projected onto the image

- Fix random order of image filenames

* a few enhancement changes to bridge gap between Inria and NS

* Fix PLY exporter and implement an PLY renderer

* Add finite filter in ply exporter

* Major changes to improve nerfstudio-gs training

* separate sh_0 and sh_rest and use different learning rates, and remove sh_rest regularization.
* disable downscale in warm-up stages
* prune the original gaussians that splits.
* change color clamp_max mechanism to be applied per ray instead of per gaussian

* Enable exporter and gsplat_render to work with color only PLY

* Prepare PR with minor changes

* Fix pyright errors

* remove gsplat ply renderer

* Address github/pwais comments

* fix an evaluation metric bug

* Use 0.1 cull_alpha_threshold for fast training

* Add scale regularization as an option (off by default)

* add continue_cull_post_densification option and some refactor

---------

Co-authored-by: Matthew Tancik <matt@lumalabs.ai>
2024-01-08 06:11:03 +00:00
Wei DingandAlexander Kristoffersen 1e23781393 Add orthophoto rendering support. (#2648)
* add orthophoto CameraType.

* test case of orthophoto camera.

* Fix the rotation errors caused by the usage of left-handed system

* Optimized ortho-cam,  it can be used with other camera models now.

* fix TypeError of coords in `test_multi_camera_type()`

---------

Co-authored-by: Alexander Kristoffersen <akristoffersen@berkeley.edu>
2024-01-07 18:17:48 -08:00
Chung Min Kim cc61cafea4 Add visible as kwarg to ViewerElements (#2730)
* Add `visible` as kwarg to ViewerElements

* Formatting
2024-01-06 15:43:26 -08:00
Matias TurkulainenandJustin Kerr 7311ff69f6 allow render path to exist (#2710)
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
2024-01-05 15:47:13 -08:00
Ruilong Li(李瑞龙) 98a21263c1 Fix periodically stuck in parallel datamanager (#2713)
* Fix sticking in parallel datamanager
2024-01-05 15:21:50 -08:00
Cyrus Vachha a8e6f8fa3f Adding docs for Instruct-GS2GS (#2702)
* Adding docs for Instruct-GS2GS

Adding documentation for Instruct-GS2GS including its page and adding it to the list of external and 3rd party methods.

* fixing blank formatting dev checks

* updating subtitle

Updating the subtitle to specify gaussian splatting
2023-12-27 21:19:19 -08:00
Abhik Ahuja 38fc72729c Fix dycheck depth and rgb rescale (#2623)
Fix depth and rgb rescale

Not sure if depth is correct since I'm not sure what the 3rd dim is for, the code runs properly though.
2023-12-27 01:07:08 +00:00
Brent Yi 7a8e31d84e Improved render panel for beta viewer (#2667)
* Continuous velocities for render splines

* Configure ruff line length

* Improved render preview

* No longer need to update camera FOV in render preview

* Details

* Polishing

* Cleanup

* Details

* Address pyright errors

* type: ignore for broken (?) Aria stub
2023-12-20 17:20:41 -08:00
Hardik Dava 4a54763323 fixed color issue in ply file (#2693)
fixed color export
2023-12-19 09:20:17 -08:00
Zhuoyang Pan f7a49563a0 Fix a bug of random quaternions (#2687)
fix a bug for random quaternions
2023-12-18 17:52:14 -08:00
Otto SeiskariandMatias Turkulainen 626441e15e Add instructions for data collection and processing with Spectacular AI tools (#2668)
* Add instructions for using Spectacular AI tools

Adds easy support for new device types: OAK-D, RealSense and
Azure Kinect, as well as an alternative pipeline for iOS.

* Grammar fixes in docs/quickstart/custom_dataset.md (Spectacular AI section)

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>

* Fix section label in the Spectacular AI section of custom_dataset.md

* Update custom_dataset.md: typo fix.

"montions" -> "motions"

---------

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
2023-12-15 18:37:27 +02:00
+5 ddedb920dc Gaussian splatting (#2521)
* Adds Gaussian Splatting integration
---------

Co-authored-by: Jake Austin jake-austin@berkeley.edu <jake-austin@berkeley.edu>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
Co-authored-by: Zhuoyang <zhuoyangpan@outlook.com>
Co-authored-by: Vickie Ye <vye1116@gmail.com>
Co-authored-by: adamrashid96 <abrashid@berkeley.edu>
Co-authored-by: Paul Wais <paulwais@gmail.com>
Co-authored-by: Gina Wu <ginawu98@gmail.com>
Co-authored-by: Alexander Kristoffersen <akristoffersen@berkeley.edu>
Co-authored-by: machenmusik <machenmusik@users.noreply.github.com>
Co-authored-by: Riley <cardiacmangoes@gmail.com>
Co-authored-by: Gina Wu <42229107+ginazhouhuiwu@users.noreply.github.com>
Co-authored-by: maturk <matiasturkulainen@gmail.com>
Co-authored-by: Guangyun-Xu <38279883+Guangyun-Xu@users.noreply.github.com>
Co-authored-by: Zhuoyang Pan <102644383+Zhuoyang-Pan@users.noreply.github.com>
2023-12-14 22:14:09 -08:00
Rohan Mathur 88873ef3ab Added support for .tif and .tiff files in equirect utils (#2613)
* added tiff to equirect utils

* added tif files
2023-12-15 01:22:20 +00:00
Tobias FischerandMatias Turkulainen 622e26b511 [Fix] Pixel Sampler setup in VanillaDataManager for Cameras incl. metadata (#2678)
fix pixel sampler setup for cameras with metadata other than fisheye crop radius

Co-authored-by: Matias Turkulainen <30566358+maturk@users.noreply.github.com>
2023-12-14 10:07:41 -08:00
Boris Feld a894c6d938 Update first_nerf.md (#2677)
Update documentation to mention that Comet visualizer can be used without the viwer
2023-12-14 08:47:51 -08:00
Justin KerrandBrent Yi 17f785758b Parity of layout between viewer beta and legacy viewer (#2639)
* alter layout of beta viewer to match old viewer

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-13 11:17:36 -08:00
Brent Yi ae00061cf5 Fix custom data links in README (#2671) 2023-12-12 22:31:12 +00:00
Chris SweeneyandBrent Yi 6ccbc2de45 Add support for Project Aria datasets. (#2617)
* Add support for Project Aria datasets.

1) An export script for processing Project Aria datasets
2) Add support for Fisheye624 cameras
3) Add support for center-crop masking and sampling

Example commands for running nerfstudio on Project Aria data:

python scripts/datasets/process_project_aria.py --vrs_file=<path_to_vrs> --mps_data_dir=<path to MPS directory> --output_dir=<nerfstudio_data_dir>

ns-train nerfacto --data <nerfstudio_data_dir>  nerfstudio-data --orientation-method none

* Clean up type errors

* Fix formatting

* Fix linter errors, import, and use absolute file paths

* Add aria subcommand to ns-process-data

* Appease ruff

* Add Aria to docs

* Fix docs, add back import try/except

* nit: path.as_posix() -> str(path)

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-12 22:18:13 +00:00
Brent YiandGina Wu 01d57b90dc Fix large dataset (> 512 images) training in viewer_beta (#2669)
* Fix large dataset (> 512 images) training in viewer_beta (#2586)

* fix index bug in update_camera_poses of viewer

* Set Ruff line lenth

---------

Co-authored-by: Gina Wu <42229107+ginazhouhuiwu@users.noreply.github.com>
2023-12-12 11:44:26 +00:00
Mohit Motwani 281f81f1ec Update cameras.py - removed duplicate lines (#2661)
The self.times was being computed twice in cameras.py in the same function which was unnecessary
2023-12-12 11:13:50 +00:00
Brent Yi 73fc3dcd63 Update dependencies for M1 Macs (#2665)
* Update dependencies for M1 Macs

* import newrawpy as rawpy

* Comment

* Add back rawpy for Linux

* Run black
2023-12-11 18:08:17 +00:00
Paul WaisandBrent Yi 390b63373d Trainer can now always save dataparser transform (#2653)
* Trainer can now always save dataparser transform

* appease the linter

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-11 13:55:36 +00:00
Alexandru Kisandmfischer 1d9cc7dd8a Update hloc and add new matcher types (#2658)
* Update hloc and add new matcher types

* Update dockerfile comment to better reflect what the command is doing

---------

Co-authored-by: mfischer <michael.fischer@spleenlab.ai>
2023-12-08 08:58:51 -08:00
omahs 4c966b9bd4 Fix typos (#2655)
* fix typo

* fix typo

* fix typos

* fix typo
2023-12-08 11:27:05 +00:00
AdamRashid96 64f0b2547b Ns-process-data bug with dev version of colmap (#2651)
* only parse digit and decimal

* formatting
2023-12-06 22:56:39 -08:00
Ethan WeberandBrent Yi 4c627edbac Encoder + MLP combo (#2063)
* adding encoder + mlp combo

* update documentation

* minor fixes

* fixed issue when tcnn isn't installed

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-01 03:16:42 +00:00
0cb410001b Fixes bug in PairPixelSampler when working with masked dataset (#2368)
* Fixes bug in PairPixelSampler when working with masked dataset

* Formatting

* Black formatting now ok.

---------

Co-authored-by: AdamRashid96 <71362382+AdamRashid96@users.noreply.github.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-01 01:11:11 +00:00
JoseandBrent Yi 49503c2175 Mps fix (#2436)
* mps bugfix

* removed docs change

* Double quotes

---------

Co-authored-by: Brent Yi <yibrenth@gmail.com>
2023-12-01 00:04:01 +00:00