mirror of
https://github.com/nerfstudio-project/nerfstudio.git
synced 2026-11-11 10:34:29 +08:00
Docs for adding new methods (#1697)
This commit is contained in:
Matthew Tancik
GitHub
parent
4e6c52c8fd
commit
7126334976
21
README.md
21
README.md
@ -196,16 +196,17 @@ ns-export pointcloud --help
|
||||
|
||||
Using an existing dataset is great, but likely you want to use your own data! We support various methods for using your own data. Before it can be used in nerfstudio, the camera location and orientations must be determined and then converted into our format using `ns-process-data`. We rely on external tools for this, instructions and information can be found in the documentation.
|
||||
|
||||
| Data | Capture Device | Requirements | `ns-process-data` Speed |
|
||||
| ------------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------------- | ----------------------- |
|
||||
| 📷 [Images](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 📹 [Video](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 🌎 [360 Data](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#360_data) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 📱 [Polycam](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#polycam-capture) | IOS with LiDAR | [Polycam App](https://poly.cam/) | 🐇 |
|
||||
| 📱 [KIRI Engine](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#kiri-capture) | IOS or Android | [KIRI Engine App](https://www.kiriengine.com/) | 🐇 |
|
||||
| 📱 [Record3D](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#record3d-capture) | IOS with LiDAR | [Record3D app](https://record3d.app/) | 🐇 |
|
||||
| 🖥 [Metashape](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 |
|
||||
| 🛠 [Custom](https://docs.nerf.studio/en/latest/quickstart/data_conventions.html) | Any | Camera Poses | 🐇 |
|
||||
| Data | Capture Device | Requirements | `ns-process-data` Speed |
|
||||
| ---------------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------------------------------- | ----------------------- |
|
||||
| 📷 [Images](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 📹 [Video](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#images-and-video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 🌎 [360 Data](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#360_data) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
| 📱 [Polycam](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#polycam-capture) | IOS with LiDAR | [Polycam App](https://poly.cam/) | 🐇 |
|
||||
| 📱 [KIRI Engine](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#kiri-capture) | IOS or Android | [KIRI Engine App](https://www.kiriengine.com/) | 🐇 |
|
||||
| 📱 [Record3D](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#record3d-capture) | IOS with LiDAR | [Record3D app](https://record3d.app/) | 🐇 |
|
||||
| 🖥 [Metashape](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 |
|
||||
| 🖥 [RealityCapture](https://docs.nerf.studio/en/latest/quickstart/custom_dataset.html#realitycapture) | Any | [RealityCapture](https://www.capturingreality.com/realitycapture) | 🐇 |
|
||||
| 🛠 [Custom](https://docs.nerf.studio/en/latest/quickstart/data_conventions.html) | Any | Camera Poses | 🐇 |
|
||||
|
||||
## 5. Advanced Options
|
||||
|
||||
|
||||
@ -118,33 +118,3 @@ Often times, you just want to play with the parameters of an existing model with
|
||||
# NOTE: the dataparser and associated configurations go at the end of the command
|
||||
ns-train {METHOD_NAME} --vis viewer {DATA_PARSER} --scale-factor 0.5
|
||||
```
|
||||
|
||||
### Extending Nerfstudio with custom methods
|
||||
In order to extend the Nerfstudio and register your own methods, you can package your code as a python package
|
||||
and register it with Nerfstudio as a `nerfstudio.method_configs` entrypoint in the `pyproject.toml` file.
|
||||
The Nerfstudio will automatically look for all registered methods and will register them to be used
|
||||
by method such as `ns-train`.
|
||||
|
||||
Here is an example:
|
||||
```python
|
||||
"""my_package/my_config.py"""
|
||||
|
||||
from nerfstudio.engine.trainer import TrainerConfig
|
||||
from nerfstudio.plugins.types import MethodSpecification
|
||||
|
||||
MyMethod = MethodSpecification(
|
||||
config=TrainerConfig(
|
||||
method_name="my-method",
|
||||
...
|
||||
),
|
||||
description="Custom description"
|
||||
)
|
||||
|
||||
"""pyproject.toml"""
|
||||
[project]
|
||||
name = "my_package"
|
||||
...
|
||||
|
||||
[project.entry-points.'nerfstudio.method_configs']
|
||||
my-method = 'my_package.my_config:MyMethod'
|
||||
```
|
||||
75
docs/developer_guides/new_methods.md
Normal file
75
docs/developer_guides/new_methods.md
Normal file
@ -0,0 +1,75 @@
|
||||
# Adding a New Method
|
||||
|
||||
Nerfstudio aims to offer researchers a codebase that they can utilize to extend and develop novel methods. Our vision is for users to establish a distinct repository that imports nerfstudio and overrides pipeline components to cater to specific functionality requirements of the new approach. If any of the new features require modifications to the core nerfstudio repository and can be generally useful, we encourage you to submit a PR to enable others to benefit from it.
|
||||
|
||||
Examples are often the best way to learn, take a look at the [LERF](https://github.com/kerrj/lerf) repository for good example of how to use nerfstudio in your projects.
|
||||
|
||||
## File Structure
|
||||
|
||||
We recommend the following file structure:
|
||||
|
||||
```
|
||||
├── my_method
|
||||
│ ├── my_config.py
|
||||
│ ├── custom_pipeline.py [optional]
|
||||
│ ├── custom_model.py [optional]
|
||||
│ ├── custom_field.py [optional]
|
||||
│ ├── custom_datamanger.py [optional]
|
||||
│ ├── ...
|
||||
├── pyproject.toml
|
||||
```
|
||||
|
||||
## Registering custom model with nerfstudio
|
||||
|
||||
In order to extend the Nerfstudio and register your own methods, you can package your code as a python package
|
||||
and register it with Nerfstudio as a `nerfstudio.method_configs` entrypoint in the `pyproject.toml` file.
|
||||
Nerfstudio will automatically look for all registered methods and will register them to be used
|
||||
by methods such as `ns-train`.
|
||||
|
||||
First create a config file:
|
||||
|
||||
```python
|
||||
"""my_method/my_config.py"""
|
||||
|
||||
from nerfstudio.engine.trainer import TrainerConfig
|
||||
from nerfstudio.plugins.types import MethodSpecification
|
||||
|
||||
MyMethod = MethodSpecification(
|
||||
config=TrainerConfig(
|
||||
method_name="my-method",
|
||||
pipeline=...
|
||||
...
|
||||
),
|
||||
description="Custom description"
|
||||
)
|
||||
```
|
||||
|
||||
Then create a `pyproject.toml` file. This is where the entrypoint to your method is set and also where you can specify additional dependencies required by your codebase.
|
||||
|
||||
```python
|
||||
"""pyproject.toml"""
|
||||
|
||||
[project]
|
||||
name = "my_method"
|
||||
|
||||
dependencies = [
|
||||
"nerfstudio" # you may want to consider pinning the version, ie "nerfstudio==0.1.19"
|
||||
]
|
||||
|
||||
[project.entry-points.'nerfstudio.method_configs']
|
||||
my-method = 'my_method.my_config:MyMethod'
|
||||
```
|
||||
|
||||
finally run the following to register the method,
|
||||
|
||||
```
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
## Running custom method
|
||||
|
||||
After registering your method you should be able to run the method with,
|
||||
|
||||
```
|
||||
ns-train my-method --data DATA_DIR
|
||||
```
|
||||
@ -75,6 +75,7 @@ nerfology/model_components/index
|
||||
:hidden:
|
||||
:caption: Developer Guides
|
||||
|
||||
developer_guides/new_methods
|
||||
developer_guides/pipelines/index
|
||||
developer_guides/viewer/viewer_overview
|
||||
developer_guides/config
|
||||
|
||||
Loading…
Reference in New Issue
Block a user