diff --git a/docs/tutorials/pipelines/creating_pipelines.ipynb b/docs/tutorials/pipelines/creating_new_methods.ipynb similarity index 98% rename from docs/tutorials/pipelines/creating_pipelines.ipynb rename to docs/tutorials/pipelines/creating_new_methods.ipynb index 368e8785e..9d9f4d1fa 100644 --- a/docs/tutorials/pipelines/creating_pipelines.ipynb +++ b/docs/tutorials/pipelines/creating_new_methods.ipynb @@ -6,7 +6,7 @@ "tags": [] }, "source": [ - "# Creating Custom Pipelines\n", + "# Creating Custom Methods\n", "\n", "> Our goal is to enable nerfstudio users to implement an entire NeRF method by only editing one file.\n", "\n", @@ -254,8 +254,6 @@ "tags": [] }, "source": [ - "# Creating pipelines from a config\n", - "\n", "Now we show how to create a pipeline from a config, which has the following form:\n", "\n", "```python\n", @@ -345,10 +343,10 @@ }, "vscode": { "interpreter": { - "hash": "100252972d4ee5286ab455ef1c778abde05b4affca9d1be78633d3797abf7e8c" + "hash": "b7b86c17c0dfd6b313529972d424f2e075cc198a46cda005757126afe8133c7b" } } }, "nbformat": 4, "nbformat_minor": 2 -} \ No newline at end of file +} diff --git a/docs/tutorials/pipelines/datamanagers.ipynb b/docs/tutorials/pipelines/datamanagers.ipynb new file mode 100644 index 000000000..a5bef2c61 --- /dev/null +++ b/docs/tutorials/pipelines/datamanagers.ipynb @@ -0,0 +1,34 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# DataManagers\n", + "\n", + "```{image} imgs/datamanagers.png\n", + ":align: center\n", + "```" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3.8.13 ('nerfactory')", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.8.13" + }, + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "b7b86c17c0dfd6b313529972d424f2e075cc198a46cda005757126afe8133c7b" + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/tutorials/pipelines/dataparsers.ipynb b/docs/tutorials/pipelines/dataparsers.ipynb new file mode 100644 index 000000000..cc92f3fe0 --- /dev/null +++ b/docs/tutorials/pipelines/dataparsers.ipynb @@ -0,0 +1,158 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# DataParsers\n", + "\n", + "```{image} imgs/dataparsers.png\n", + ":align: center\n", + "```\n", + "\n", + "## What is a DataParser?\n", + "\n", + "The dataparser returns `DataparserOutputs`, which puts all the various datasets into a common format.\n", + "\n", + "```python\n", + "@dataclass\n", + "class DataparserOutputs:\n", + " \"\"\"Dataset inputs for the image dataset and the ray generator.\"\"\"\n", + "\n", + " image_filenames: List[Path]\n", + " \"\"\"Filenames for the images.\"\"\"\n", + " cameras: Cameras\n", + " \"\"\"Camera object storing collection of camera information in dataset\"\"\"\n", + " alpha_color: Optional[TensorType[3]] = None\n", + " \"\"\"color of dataset background\"\"\"\n", + " scene_box: SceneBox = SceneBox()\n", + " \"\"\"scene box of dataset. Could be used to bound the scene, or to provide the scene scale depending on model.\"\"\"\n", + " ...\n", + "\n", + "@dataclass\n", + "class DataParser:\n", + "\n", + " @abstractmethod\n", + " def _generate_dataset_inputs(self, split: str = \"train\") -> DataparserOutputs:\n", + " \"\"\"Abstract method that returns the dataparser outputs for the given split.\n", + "\n", + " Args:\n", + " split: Which dataset split to generate data for (train/eval/test).\n", + "\n", + " Returns:\n", + " DataparserOutputs for the specified dataset and split.\n", + " \"\"\"\n", + "```\n", + "\n", + "## Example\n", + "\n", + "Here is an example where we implement a DataParser for our Nerfstudio data format.\n", + "\n", + "```python\n", + "@dataclass\n", + "class NerfstudioDataParserConfig(DataParserConfig):\n", + " \"\"\"Nerfstudio dataset config\"\"\"\n", + "\n", + " _target: Type = field(default_factory=lambda: Nerfstudio)\n", + " \"\"\"Target class to instantiate.\"\"\"\n", + " data_directory: Path = Path(\"data/nerfstudio/poster\")\n", + " \"\"\"Directory specifying location of data.\"\"\"\n", + " scale_factor: float = 1.0\n", + " \"\"\"How much to scale the camera origins by.\"\"\"\n", + " downscale_factor: int = 1\n", + " \"\"\"How much to downscale images.\"\"\"\n", + " scene_scale: float = 1.0\n", + " \"\"\"How much to scale the region of interest by.\"\"\"\n", + " orientation_method: Literal[\"pca\", \"up\"] = \"up\"\n", + " \"\"\"The method to use for orientation.\"\"\"\n", + " train_split_percentage: float = 0.9\n", + " \"\"\"The percent of images to use for training. The remaining images are for eval.\n", + " \"\"\"\n", + "\n", + "@dataclass\n", + "class Nerfstudio(DataParser):\n", + " \"\"\"Nerfstudio Dataset\"\"\"\n", + "\n", + " config: NerfstudioDataParserConfig\n", + "\n", + " def _generate_dataset_inputs(self, split=\"train\") -> DataParserOutputs:\n", + " \"\"\"Returns DataParserOutputs.\"\"\"\n", + "```\n", + "\n", + "## Our Implementations\n", + "\n", + "###### Nerfstudio\n", + "\n", + "This is our custom dataparser.\n", + "\n", + "```{button-link} https://github.com/nerfstudio-project/nerfstudio/blob/master/nerfstudio/data/dataparsers/nerfstudio_dataparser.py\n", + ":color: primary\n", + ":outline:\n", + "See the code!\n", + "```\n", + "\n", + "###### Blender\n", + "\n", + "We support the synthetic Blender dataset from the original NeRF paper.\n", + "\n", + "```{button-link} https://github.com/nerfstudio-project/nerfstudio/blob/master/nerfstudio/data/dataparsers/blender_dataparser.py\n", + ":color: primary\n", + ":outline:\n", + "See the code!\n", + "```\n", + "\n", + "###### Instant NGP\n", + "\n", + "This supports the Instant NGP dataset.\n", + "\n", + "```{button-link} https://github.com/nerfstudio-project/nerfstudio/blob/master/nerfstudio/data/dataparsers/instant_ngp_dataparser.py\n", + ":color: primary\n", + ":outline:\n", + "See the code!\n", + "```\n", + "\n", + "###### MipNeRF 360\n", + "\n", + "```{button-link} https://github.com/nerfstudio-project/nerfstudio/blob/master/nerfstudio/data/dataparsers/mipnerf_dataparser.py\n", + ":color: primary\n", + ":outline:\n", + "See the code!\n", + "```\n", + "\n", + "###### Record3D\n", + "\n", + "This dataparser can use recorded data from a >= iPhone 12 Pro using the Record3D app. Record a video and export with the `EXR + JPG sequence` format. Unzip export and `rgb` folder before training.\n", + "\n", + "```{button-link} https://github.com/nerfstudio-project/nerfstudio/blob/master/nerfstudio/data/dataparsers/record3d_dataparser.py\n", + ":color: primary\n", + ":outline:\n", + "See the code!\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3.8.13 ('nerfactory')", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.8.13" + }, + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "b7b86c17c0dfd6b313529972d424f2e075cc198a46cda005757126afe8133c7b" + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/tutorials/pipelines/imgs/datamanagers.png b/docs/tutorials/pipelines/imgs/datamanagers.png new file mode 100644 index 000000000..c1c881b77 Binary files /dev/null and b/docs/tutorials/pipelines/imgs/datamanagers.png differ diff --git a/docs/tutorials/pipelines/imgs/dataparsers.png b/docs/tutorials/pipelines/imgs/dataparsers.png new file mode 100644 index 000000000..d7ebe9710 Binary files /dev/null and b/docs/tutorials/pipelines/imgs/dataparsers.png differ diff --git a/docs/tutorials/pipelines/imgs/models.png b/docs/tutorials/pipelines/imgs/models.png new file mode 100644 index 000000000..e637a1359 Binary files /dev/null and b/docs/tutorials/pipelines/imgs/models.png differ diff --git a/docs/tutorials/pipelines/imgs/pipelines.png b/docs/tutorials/pipelines/imgs/pipelines.png new file mode 100644 index 000000000..a41d187ed Binary files /dev/null and b/docs/tutorials/pipelines/imgs/pipelines.png differ diff --git a/docs/tutorials/pipelines/imgs/pipelines_overview.png b/docs/tutorials/pipelines/imgs/pipelines_overview.png new file mode 100644 index 000000000..9ce62a8ec Binary files /dev/null and b/docs/tutorials/pipelines/imgs/pipelines_overview.png differ diff --git a/docs/tutorials/pipelines/index.rst b/docs/tutorials/pipelines/index.rst index de061bb58..023f39068 100644 --- a/docs/tutorials/pipelines/index.rst +++ b/docs/tutorials/pipelines/index.rst @@ -1,10 +1,19 @@ -Creating Custom Pipelines +Pipelines Overview ------------------------- Here we show you how to create a custom pipeline. +# TODO: show a figure of what a pipeline is here + +.. image:: imgs/pipelines_overview.png + :width: 400 + :align: center + .. toctree:: :maxdepth: 1 - creating_pipelines.ipynb - creating_models.ipynb \ No newline at end of file + dataparsers.ipynb + datamanagers.ipynb + models.ipynb + pipelines.ipynb + creating_new_methods.ipynb \ No newline at end of file diff --git a/docs/tutorials/pipelines/creating_models.ipynb b/docs/tutorials/pipelines/models.ipynb similarity index 97% rename from docs/tutorials/pipelines/creating_models.ipynb rename to docs/tutorials/pipelines/models.ipynb index e2413b3d1..8fdf48530 100644 --- a/docs/tutorials/pipelines/creating_models.ipynb +++ b/docs/tutorials/pipelines/models.ipynb @@ -4,7 +4,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Developing Custom Models\n", + "# Models\n", + "\n", + "```{image} imgs/models.png\n", + ":align: center\n", + "```\n", "\n", "In this quick tour, we will walk you through the core of training and building any NeRFs with nerfstudio.\n", "\n", @@ -201,7 +205,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3.8.12 ('nerfactory')", + "display_name": "Python 3.8.13 ('nerfactory')", "language": "python", "name": "python3" }, @@ -215,12 +219,12 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.8.13" }, "orig_nbformat": 4, "vscode": { "interpreter": { - "hash": "34c28001ff35fb390494047002768a8182dcf55b1b11415165e62ea61557ab83" + "hash": "b7b86c17c0dfd6b313529972d424f2e075cc198a46cda005757126afe8133c7b" } } }, diff --git a/docs/tutorials/pipelines/pipelines.ipynb b/docs/tutorials/pipelines/pipelines.ipynb new file mode 100644 index 000000000..ab79f9336 --- /dev/null +++ b/docs/tutorials/pipelines/pipelines.ipynb @@ -0,0 +1,39 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Pipelines\n", + "\n", + "```{image} imgs/pipelines.png\n", + ":align: center\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3.8.13 ('nerfactory')", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.8.13" + }, + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "b7b86c17c0dfd6b313529972d424f2e075cc198a46cda005757126afe8133c7b" + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}