diff --git a/README.md b/README.md index a90020772..fefe92af4 100644 --- a/README.md +++ b/README.md @@ -96,14 +96,14 @@ pip install nerfstudio If you would want the latest and greatest: ```bash -git clone git@github.com:plenoptix/nerfstudio.git +git clone git@github.com:nerfstudio-project/nerfstudio.git cd nerfstudio pip install -e . ``` ## 2. Setting up the data -Download the original NeRF Blender dataset. We support the major datasets and allow users to create their own dataset, described in detail [here](https://plenoptix-nerfstudio.readthedocs-hosted.com/en/latest/tutorials/data/index.html). +Download the original NeRF Blender dataset. We support the major datasets and allow users to create their own dataset, described in detail [here](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html). ```bash ns-download-data --dataset=blender @@ -111,7 +111,8 @@ ns-download-data --dataset=nerfstudio --capture=poster ``` ### 2.x Using custom data -If you have custom data in the form of a video or folder of images, we've provided some [COLMAP](https://colmap.github.io/) and [FFmpeg](https://ffmpeg.org/download.html) scripts to help you process your data so it is compatible with nerfstudio. + +If you have custom data in the form of a video or folder of images, we've provided some [COLMAP](https://colmap.github.io/) and [FFmpeg](https://ffmpeg.org/download.html) scripts to help you process your data so it is compatible with nerfstudio. After installing both software, you can process your data via: @@ -168,31 +169,30 @@ After your model has trained, you can headlessly render out a video of the scene ns-render --load-config=outputs/data-nerfstudio-poster/nerfacto/{TIMESTAMP}/config.yml --traj=spiral --output-path=output.mp4 ``` - # Learn More -And that's it for getting started with the basics of nerfstudio. +And that's it for getting started with the basics of nerfstudio. If you're interested in learning more on how to create your own pipelines, develop with the viewer, run benchmarks, and more, please check out some of the qucklinks below or visit our [documentation](https://docs.nerf.studio/en/latest/) directly. -| Section | Description | -| --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -| [Documentation](https://docs.nerf.studio/en/latest/) | Full API documentation and tutorials | -| [Viewer](https://viewer.nerf.studio/) | Home page for our web viewer | -| ๐ŸŽ’ **Educational** | -| [Model Descriptions](https://docs.nerf.studio/en/latest/guides/index.html) | Description of all the models supported by nerfstudio and explanations of component parts. | -| [Component Descriptions](https://docs.nerf.studio/en/latest/nerfology/model_components/index.html) | Interactive notebooks that explain notable/commonly used modules in various models. | -| ๐Ÿƒ **Tutorials** | -| [Getting Started](https://docs.nerf.studio/en/latest/quickstart/installation.html) | A more in-depth guide on how to get started with nerfstudio from installation to contributing. | -| [Using the Viewer](https://docs.nerf.studio/en/latest/quickstart/viewer_quickstart.html) | A quick demo video on how to navigate the viewer. | -| ๐Ÿ’ป **For Developers** | -| [Creating pipelines](https://docs.nerf.studio/en/latest/developer_guides/pipelines/index.html) | Learn how to easily build new neural rendering pipelines by using and/or implementing new modules. | -| [Creating datasets](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html) | Have a new dataset? Learn how to run it with nerfstudio. | | -| [Contributing](https://docs.nerf.studio/en/latest/reference/contributing.html) | Walk-through for how you can start contributing now. | -| ๐Ÿ’– **Community** | -| [Discord](https://discord.gg/NHGtYRAW) | Join our community to discuss more. We would love to hear from you! | -| [Twitter](#) | Follow us on Twitter to see cool updates and announcements | -| [TikTok](#) | Comming soon! Follow us on TikTok to see some of our fan favorite results | +| Section | Description | +| -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | --- | +| [Documentation](https://docs.nerf.studio/en/latest/) | Full API documentation and tutorials | +| [Viewer](https://viewer.nerf.studio/) | Home page for our web viewer | +| ๐ŸŽ’ **Educational** | +| [Model Descriptions](https://docs.nerf.studio/en/latest/guides/index.html) | Description of all the models supported by nerfstudio and explanations of component parts. | +| [Component Descriptions](https://docs.nerf.studio/en/latest/nerfology/model_components/index.html) | Interactive notebooks that explain notable/commonly used modules in various models. | +| ๐Ÿƒ **Tutorials** | +| [Getting Started](https://docs.nerf.studio/en/latest/quickstart/installation.html) | A more in-depth guide on how to get started with nerfstudio from installation to contributing. | +| [Using the Viewer](https://docs.nerf.studio/en/latest/quickstart/viewer_quickstart.html) | A quick demo video on how to navigate the viewer. | +| ๐Ÿ’ป **For Developers** | +| [Creating pipelines](https://docs.nerf.studio/en/latest/developer_guides/pipelines/index.html) | Learn how to easily build new neural rendering pipelines by using and/or implementing new modules. | +| [Creating datasets](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html) | Have a new dataset? Learn how to run it with nerfstudio. | | +| [Contributing](https://docs.nerf.studio/en/latest/reference/contributing.html) | Walk-through for how you can start contributing now. | +| ๐Ÿ’– **Community** | +| [Discord](https://discord.gg/NHGtYRAW) | Join our community to discuss more. We would love to hear from you! | +| [Twitter](#) | Follow us on Twitter to see cool updates and announcements | +| [TikTok](#) | Comming soon! Follow us on TikTok to see some of our fan favorite results | # Supported Features @@ -207,7 +207,7 @@ We provide the following support strucutures to make life easier for getting sta - And more! - :pencil2: Support for multiple logging interfaces (Tensorboard, Wandb), code profiling, and other built-in debugging tools - :chart_with_upwards_trend: Easy-to-use benchmarking scripts on the Blender dataset -- :iphone: Full pipeline support (w/ Colmap or Record3D) for going from a video on your phone to a full 3D render. +- :iphone: Full pipeline support (w/ Colmap or Record3D) for going from a video on your phone to a full 3D render. # See what's possible diff --git a/docs/developer_guides/pipelines/models.md b/docs/developer_guides/pipelines/models.md index 02601163a..57da37697 100644 --- a/docs/developer_guides/pipelines/models.md +++ b/docs/developer_guides/pipelines/models.md @@ -14,7 +14,7 @@ ## What is a Model? -A Model is probably what you think of when you think of a NeRF paper. Often the phrases "Model" and "Method" are used interchangeably and for this reason, our [implemented models](https://docs.nerf.studio/en/latest/nerfology/models/index.html) are actually methods. However, most of the implementations details of a method happen in the model code, as you'll see in this section. +A Model is probably what you think of when you think of a NeRF paper. Often the phrases "Model" and "Method" are used interchangeably and for this reason, our [implemented models](/nerfology/models/index) are actually methods. However, most of the implementations details of a method happen in the model code, as you'll see in this section. A model, at a high level, takes in regions of space described by RayBundle objects, samples points along these rays, and returns rendered values for each ray. So, let's take a look at what it takes to create your own model! diff --git a/docs/developer_guides/viewer/viewer_notebook.ipynb b/docs/developer_guides/viewer/viewer_notebook.ipynb index 5e74dd7da..d791ac338 100644 --- a/docs/developer_guides/viewer/viewer_notebook.ipynb +++ b/docs/developer_guides/viewer/viewer_notebook.ipynb @@ -50,7 +50,7 @@ "source": [ "## Open the Nerfviewer client\n", "\n", - "Open the Nerfviewer client by visiting http://nerfviewer.plenoptix.ai/ or running the code locally. *We refer to this document for running the code locally.* You should see a GUI like the following:\n", + "Open the Nerfviewer client by visiting http://viewer.nerf.studio/ or running the code locally. *We refer to this document for running the code locally.* You should see a GUI like the following:\n", "\n", "```{image} imgs/nerfviewer.png\n", ":align: center\n", diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..5dc2cfac3 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,109 @@ +
+ +```{image} _static/imgs/logo.png +:width: 400 +:align: center +:alt: nerfstudio +:class: only-light +``` + +```{image} _static/imgs/logo-dark.png +:width: 400 +:align: center +:alt: nerfstudio +:class: only-dark +``` + +
+ +Nerfstudio provides a simple API that allows for a simplified end-to-end process of creating, training, and testing NeRFs. +The library supports a **more interpretable implementation of NeRFs by modularizing each component.** +With more modular NeRFs, we hope to create a more user-friendly experience in exploring the technology. +Nerfstudio is a contributer friendly repo with the goal of buiding a community where users can more easily build upon each other's contributions. + +It's as simple as plug and play with nerfstudio! + +Ontop of our API, we are commited to providing learning resources to help you understand the basics of (if you're just getting start), and keep up-to-date with (if you're a seasoned veteran) all things NeRF. +As researchers, we know just how hard it is to get onboarded with this next-gen technology. So we're here to help with tutorials, documentation, and more! + +Finally, have feature requests? Want to add your brand-spankin'-new NeRF model? Have a new dataset? **We welcome any and all [contributions](reference/contributing)!** +Please do not hesitate to reach out to the nerfstudio team with any questions via [Discord](https://discord.gg/NHGtYRAW). + +We hope nerfstudio enables you to build faster ๐Ÿ”จ learn together ๐Ÿ“š and contribute to our NeRF community ๐Ÿ’–. + +## Contents + +```{toctree} +:hidden: +:caption: Getting Started + +quickstart/installation +quickstart/first_nerf +quickstart/custom_dataset +quickstart/viewer_quickstart +Contributing +``` + +```{toctree} +:hidden: +:caption: NeRFology + +nerfology/models/index +nerfology/model_components/index +nerfology/dataloader_components/index +``` + +```{toctree} +:hidden: +:caption: Developer Guides + +developer_guides/pipelines/index +developer_guides/viewer/index +developer_guides/config +developer_guides/logging_profiling +developer_guides/benchmarking +``` + +```{toctree} +:hidden: +:caption: Reference + +reference/cli/index +reference/api/index +``` + +This documentation is organized into 3 parts: + +- **๐Ÿƒโ€โ™€๏ธ Getting Started**: a great place to start if you are new to nerfstudio. Contains a quick tour, installation, and an overview of the core structures that will allow you to get up and running with nerfstudio. +- **๐Ÿงช Nerfology**: want to learn more about the tech itself? We're here to help with our educational guides. We've provided some interactive notebooks that walk you through what each component is all about. +- **๐Ÿค“ Developer Guides**: describes all of the components and additional support we provide to help you construct, train, and debug your NeRFs. Learn how to set up a model pipeline, use the viewer, create a custom config, and more. +- **๐Ÿ“š Reference**: describes each class and function. Develop a better understanding of the core of our technology and terminology. This section includes descriptions of each module and component in the codebase. + +## Supported Models + +- **Nerfacto**: our defacto NeRF model combines modules focused on quality with modules focused on faster rendering. Nerfstudio easily lets us experiment with the best of both worlds! +- [NeRF](https://www.matthewtancik.com/nerf): Representing Scenes as Neural Radiance Fields for View Synthesis +- [Instant NGP](https://nvlabs.github.io/instant-ngp/): Instant Neural Graphics Primitives with a Multiresolution Hash Encoding +- [Mipnerf-360](https://jonbarron.info/mipnerf360/): Unbounded Anti-Aliased Neural Radiance Fields +- [Mipnerf](https://jonbarron.info/mipnerf/): A Multiscale Representation for Anti-Aliasing Neural Radiance Fields +- [NerfW](https://nerf-w.github.io/): Neural Radiance Fields for Unconstrained Photo Collections +- [Semantic NeRF](https://shuaifengzhi.com/Semantic-NeRF/): In-Place Scene Labelling and Understanding with Implicit Scene Representation +- [TensoRF](https://apchenstu.github.io/TensoRF/): Tensorial Radiance Fields + +We'll be constantly growing this list! So make sure to check back in to see our updates. + +**Eager to contribute?** We'd love to see you use nerfstudio in implementing new (or even existing) models! Feel free to contact us directly or view our [Contributor's Guide](reference/contributing) to see how you can get your model on this list! + +## Quicklinks + +| | | +| ---------------------------------------------------------- | ---------------------- | +| [Github](https://github.com/nerfstudio-project/nerfstudio) | Official Github Repo | +| [Discord](https://discord.com/invite/NHGtYRAW) | Join Discord Community | +| [Viewer](https://viewer.nerf.studio/) | Web-based Nerf Viewer | + +## Indices and Tables + +- {ref}`genindex` +- {ref}`modindex` +- {ref}`search` diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 66831d446..000000000 --- a/docs/index.rst +++ /dev/null @@ -1,124 +0,0 @@ - -| - -.. image:: _static/imgs/logo.png - :width: 400 - :align: center - :alt: nerfstudio - :class: only-light - -.. image:: _static/imgs/logo-dark.png - :width: 400 - :align: center - :alt: nerfstudio - :class: only-dark - - -| - - -Nerfstudio provides a simple API that allows for a simplified end-to-end process of creating, training, and testing NeRFs. -The library supports a **more interpretable implementation of NeRFs by modularizing each component.** -With more modular NeRFs, we hope to create a more user-friendly experience in exploring the technology. -Nerfstudio is a contributer friendly repo with the goal of buiding a community where users can more easily build upon each other's contributions. - -It's as simple as plug and play with nerfstudio! - -Ontop of our API, we are commited to providing learning resources to help you understand the basics of (if you're just getting start), and keep up-to-date with (if you're a seasoned veteran) all things NeRF. -As researchers, we know just how hard it is to get onboarded with this next-gen technology. So we're here to help with tutorials, documentation, and more! - -Finally, have feature requests? Want to add your brand-spankin'-new NeRF model? Have a new dataset? **We welcome any and all [contributions](https://docs.nerf.studio/en/latest/reference/contributing.html)!** -Please do not hesitate to reach out to the nerfstudio team with any questions via [Discord](https://discord.gg/NHGtYRAW). - -We hope nerfstudio enables you to build faster ๐Ÿ”จ learn together ๐Ÿ“š and contribute to our NeRF community ๐Ÿ’–. - -| - - -Contents -""""""""""""""""""""""""""""" - -.. toctree:: - :hidden: - :caption: Getting Started - - quickstart/installation - quickstart/first_nerf - quickstart/custom_dataset - quickstart/viewer_quickstart - Contributing - -.. toctree:: - :hidden: - :caption: NeRFology - - nerfology/models/index - nerfology/model_components/index - nerfology/dataloader_components/index - -.. toctree:: - :hidden: - :caption: Developer Guides - - developer_guides/pipelines/index - developer_guides/viewer/index - developer_guides/config - developer_guides/logging_profiling - developer_guides/benchmarking - -.. toctree:: - :hidden: - :caption: Reference - - reference/cli/index - reference/api/index - - - -This documentation is organized into 3 parts: - -* **๐Ÿƒโ€โ™€๏ธ Getting Started**: a great place to start if you are new to nerfstudio. Contains a quick tour, installation, and an overview of the core structures that will allow you to get up and running with nerfstudio. -* **๐Ÿงช Nerfology**: want to learn more about the tech itself? We're here to help with our educational guides. We've provided some interactive notebooks that walk you through what each component is all about. -* **๐Ÿค“ Developer Guides**: describes all of the components and additional support we provide to help you construct, train, and debug your NeRFs. Learn how to set up a model pipeline, use the viewer, create a custom config, and more. -* **๐Ÿ“š Reference**: describes each class and function. Develop a better understanding of the core of our technology and terminology. This section includes descriptions of each module and component in the codebase. - - -| - - -Supported Models -""""""""""""""""""""""""""""" - -* **Nerfacto**: our defacto NeRF model combines modules focused on quality with modules focused on faster rendering. Nerfstudio easily lets us experiment with the best of both worlds! -* `NeRF `_: Representing Scenes as Neural Radiance Fields for View Synthesis -* `Instant NGP `_: Instant Neural Graphics Primitives with a Multiresolution Hash Encoding -* `Mipnerf-360 `_: Unbounded Anti-Aliased Neural Radiance Fields -* `Mipnerf `_: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields -* `NerfW `_: Neural Radiance Fields for Unconstrained Photo Collections -* `Semantic NeRF `_: In-Place Scene Labelling and Understanding with Implicit Scene Representation -* `TensoRF `_: Tensorial Radiance Fields - -We'll be constantly growing this list! So make sure to check back in to see our updates. - -**Eager to contribute?** We'd love to see you use nerfstudio in implementing new (or even existing) models! Feel free to contact us directly or view our `Contributor's Guide `_ to see how you can get your model on this list! - -Quicklinks -""""""""""""""""""""""""""""" - -.. table:: - :align: left - :widths: auto - - ============================================================ ====================== - `Github `_ Official Github Repo - `Discord `_ Join Discord Community - `Viewer `_ Web-based Nerf Viewer - ============================================================ ====================== - - -Indices and Tables -""""""""""""""""""""""""""""" - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/quickstart/installation.md b/docs/quickstart/installation.md index 6728a291a..96de6e61d 100644 --- a/docs/quickstart/installation.md +++ b/docs/quickstart/installation.md @@ -32,7 +32,7 @@ pip install nerfstudio If you want the latest and greatest: ```bash -git clone git@github.com:plenoptix/nerfstudio.git +git clone git@github.com:nerfstudio-project/nerfstudio.git cd nerfstudio pip install -e . @@ -86,7 +86,6 @@ This can be fixed by upgrading pip to the latest version: python -m pip install --upgrade pip ``` - (cuda-sources-error)= #### Runtime errors: "len(sources) > 0", "ctype = \_C.ContractionType(type.value) ; TypeError: 'NoneType' object is not callable". @@ -100,4 +99,4 @@ This is a problem with not being able to detect the correct CUDA version, and ca export CUDA_HOME=/usr/local/cuda export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64 export PATH=$PATH:$CUDA_HOME/bin -``` \ No newline at end of file +``` diff --git a/nerfstudio/models/semantic_nerfw.py b/nerfstudio/models/semantic_nerfw.py index ac11c0b4d..8dd866935 100644 --- a/nerfstudio/models/semantic_nerfw.py +++ b/nerfstudio/models/semantic_nerfw.py @@ -1,4 +1,4 @@ -# Copyright 2022 The Plenoptix Team. All rights reserved. +# Copyright 2022 The nerfstudio Team. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -51,7 +51,7 @@ from nerfstudio.model_components.renderers import ( from nerfstudio.model_components.scene_colliders import NearFarCollider from nerfstudio.models.base_model import Model from nerfstudio.models.nerfacto import NerfactoModelConfig -from nerfstudio.utils import colormaps, colors +from nerfstudio.utils import colormaps @dataclass @@ -121,8 +121,7 @@ class SemanticNerfWModel(Model): ) # renderers - background_color = None if self.config.randomize_background else colors.WHITE - self.renderer_rgb = RGBRenderer(background_color=background_color) + self.renderer_rgb = RGBRenderer(background_color=self.config.background_color) self.renderer_accumulation = AccumulationRenderer() self.renderer_depth = DepthRenderer() self.renderer_uncertainty = UncertaintyRenderer()