diff --git a/README.md b/README.md index ddd0873e0..57d77724b 100644 --- a/README.md +++ b/README.md @@ -196,16 +196,17 @@ ns-export pointcloud --help Using an existing dataset is great, but likely you want to use your own data! We support various methods for using your own data. Before it can be used in nerfstudio, the camera location and orientations must be determined and then converted into our format using `ns-process-data`. We rely on external tools for this, instructions and information can be found in the documentation. -| Data | Capture Device | Requirements | `ns-process-data` Speed | -| ------------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------------- | ----------------------- | -| 📷 [Images](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | -| 📹 [Video](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | -| 🌎 [360 Data](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#360_data) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | -| 📱 [Polycam](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#polycam-capture) | IOS with LiDAR | [Polycam App](https://poly.cam/) | 🐇 | -| 📱 [KIRI Engine](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#kiri-capture) | IOS or Android | [KIRI Engine App](https://www.kiriengine.com/) | 🐇 | -| 📱 [Record3D](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#record3d-capture) | IOS with LiDAR | [Record3D app](https://record3d.app/) | 🐇 | -| 🖥 [Metashape](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 | -| 🛠 [Custom](https://docs.nerf.studio/en/latest/quickstart/data_conventions.html) | Any | Camera Poses | 🐇 | +| Data | Capture Device | Requirements | `ns-process-data` Speed | +| ---------------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------------------------------- | ----------------------- | +| 📷 [Images](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | +| 📹 [Video](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | +| 🌎 [360 Data](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#360_data) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 | +| 📱 [Polycam](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#polycam-capture) | IOS with LiDAR | [Polycam App](https://poly.cam/) | 🐇 | +| 📱 [KIRI Engine](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#kiri-capture) | IOS or Android | [KIRI Engine App](https://www.kiriengine.com/) | 🐇 | +| 📱 [Record3D](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#record3d-capture) | IOS with LiDAR | [Record3D app](https://record3d.app/) | 🐇 | +| 🖥 [Metashape](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 | +| 🖥 [RealityCapture](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#realitycapture) | Any | [RealityCapture](https://www.capturingreality.com/realitycapture) | 🐇 | +| 🛠 [Custom](https://docs.nerf.studio/en/latest/quickstart/data_conventions.html) | Any | Camera Poses | 🐇 | ## 5. Advanced Options diff --git a/docs/developer_guides/config.md b/docs/developer_guides/config.md index 29d093f9d..bf216278f 100644 --- a/docs/developer_guides/config.md +++ b/docs/developer_guides/config.md @@ -118,33 +118,3 @@ Often times, you just want to play with the parameters of an existing model with # NOTE: the dataparser and associated configurations go at the end of the command ns-train {METHOD_NAME} --vis viewer {DATA_PARSER} --scale-factor 0.5 ``` - -### Extending Nerfstudio with custom methods -In order to extend the Nerfstudio and register your own methods, you can package your code as a python package -and register it with Nerfstudio as a `nerfstudio.method_configs` entrypoint in the `pyproject.toml` file. -The Nerfstudio will automatically look for all registered methods and will register them to be used -by method such as `ns-train`. - -Here is an example: -```python -"""my_package/my_config.py""" - -from nerfstudio.engine.trainer import TrainerConfig -from nerfstudio.plugins.types import MethodSpecification - -MyMethod = MethodSpecification( - config=TrainerConfig( - method_name="my-method", - ... - ), - description="Custom description" -) - -"""pyproject.toml""" -[project] -name = "my_package" -... - -[project.entry-points.'nerfstudio.method_configs'] -my-method = 'my_package.my_config:MyMethod' -``` \ No newline at end of file diff --git a/docs/developer_guides/new_methods.md b/docs/developer_guides/new_methods.md new file mode 100644 index 000000000..c0484ca9e --- /dev/null +++ b/docs/developer_guides/new_methods.md @@ -0,0 +1,75 @@ +# Adding a New Method + +Nerfstudio aims to offer researchers a codebase that they can utilize to extend and develop novel methods. Our vision is for users to establish a distinct repository that imports nerfstudio and overrides pipeline components to cater to specific functionality requirements of the new approach. If any of the new features require modifications to the core nerfstudio repository and can be generally useful, we encourage you to submit a PR to enable others to benefit from it. + +Examples are often the best way to learn, take a look at the [LERF](https://github.com/kerrj/lerf) repository for good example of how to use nerfstudio in your projects. + +## File Structure + +We recommend the following file structure: + +``` +├── my_method +│ ├── my_config.py +│ ├── custom_pipeline.py [optional] +│ ├── custom_model.py [optional] +│ ├── custom_field.py [optional] +│ ├── custom_datamanger.py [optional] +│ ├── ... +├── pyproject.toml +``` + +## Registering custom model with nerfstudio + +In order to extend the Nerfstudio and register your own methods, you can package your code as a python package +and register it with Nerfstudio as a `nerfstudio.method_configs` entrypoint in the `pyproject.toml` file. +Nerfstudio will automatically look for all registered methods and will register them to be used +by methods such as `ns-train`. + +First create a config file: + +```python +"""my_method/my_config.py""" + +from nerfstudio.engine.trainer import TrainerConfig +from nerfstudio.plugins.types import MethodSpecification + +MyMethod = MethodSpecification( + config=TrainerConfig( + method_name="my-method", + pipeline=... + ... + ), + description="Custom description" +) +``` + +Then create a `pyproject.toml` file. This is where the entrypoint to your method is set and also where you can specify additional dependencies required by your codebase. + +```python +"""pyproject.toml""" + +[project] +name = "my_method" + +dependencies = [ + "nerfstudio" # you may want to consider pinning the version, ie "nerfstudio==0.1.19" +] + +[project.entry-points.'nerfstudio.method_configs'] +my-method = 'my_method.my_config:MyMethod' +``` + +finally run the following to register the method, + +``` +pip install -e . +``` + +## Running custom method + +After registering your method you should be able to run the method with, + +``` +ns-train my-method --data DATA_DIR +``` diff --git a/docs/index.md b/docs/index.md index b373097e5..d28393883 100644 --- a/docs/index.md +++ b/docs/index.md @@ -75,6 +75,7 @@ nerfology/model_components/index :hidden: :caption: Developer Guides +developer_guides/new_methods developer_guides/pipelines/index developer_guides/viewer/viewer_overview developer_guides/config