Reorganize documentation (#631)

restructure_docs
This commit is contained in:
Matthew Tancik authored and GitHub committed 2022-10-03 12:59:46 -07:00
1 parent 5a76b955cd
commit d7c34ca2bd
68 files changed
+87 -148

No files matched your search

+100
View File
@@ -0,0 +1,100 @@
# Using Custom Data
Training model on existing datasets is only so fun. If you would like to train on self captured data you will need to process the data into an existing format. Specifically we need to know the camera poses for each image. [COLMAP](https://github.com/colmap/colmap) is a standard tool for extracting poses. It is possible to use other methods like [SLAM](https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping) or hardware recorded poses. We intend to add documentation for these other methods in the future.
## nerfstudio dataset
To assist running on custom data we have a script that will process a video or folder of images into a format that is compatible with nerfstudio. We use [COLMAP](https://colmap.github.io) and [FFmpeg](https://ffmpeg.org/download.html) in our data processing script, please have these installed. We have provided a quickstart to installing COLMAP below, FFmpeg can be downloaded from [here](https://ffmpeg.org/download.html)
To process your data run:
```bash
ns-process-data --data FOLDER_OR_VIDEO --output-dir PROCESSED_DATA_DIR
```
A full set of arguments can be found {doc}`here</reference/cli/ns_process_data>`.
:::{admonition} Tip
:class: info
- COLMAP can be finicky. Try your best to capture overlapping, non-blurry images.
:::
### Training on your data
Simply specify that you are using the `nerfstudio` dataparser and point the data directory to your processed data.
```bash
python scripts/train.py instant-ngp nerfstudio-data --pipeline.datamanager.dataparser.data-directory PROCESSED_DATA_DIR
```
### Installing COLMAP
The COLMAP maintainers recommend using VCPKG to install COLAMP. If the following command do not work, please refer to the [COLMAP installation guide](https://colmap.github.io/install.html) for additional installation methods.
::::::{tab-set}
:::::{tab-item} Linux
::::{tab-set}
:::{tab-item} CUDA
```bash
git clone https://github.com/microsoft/vcpkg
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install colmap[cuda]:x64-linux
```
:::
:::{tab-item} CPU
```bash
git clone https://github.com/microsoft/vcpkg
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install colmap:x64-linux
```
:::
::::
:::::
:::::{tab-item} OSX
```bash
git clone https://github.com/microsoft/vcpkg
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install colmap
```
:::::
:::::{tab-item} Windows
::::{tab-set}
:::{tab-item} CUDA
```bash
git clone https://github.com/microsoft/vcpkg
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install colmap[cuda]:x64-windows
```
:::
:::{tab-item} CPU
```bash
git clone https://github.com/microsoft/vcpkg
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install colmap:x64-windows
```
:::
::::
:::::
::::::
+114
View File
@@ -0,0 +1,114 @@
# Training First Model
## Downloading data
Download the original NeRF Blender dataset. We support the major datasets and allow users to create their own dataset, described in detail [here TODO].
```
ns-download-data --dataset=blender
ns-download-data --dataset=nerfstudio --capture=poster
```
Use `--help` to view all currently available datasets. The resulting script should download and unpack the dataset as follows:
```
|─ nerfstudio/
├─ data/
| ├─ blender/
| ├─ fern/
| ├─ lego/
...
|- <dataset_format>/
|- <scene>
...
```
## Training a model
To run with all the defaults, e.g. vanilla nerf method with the blender lego image
Run a vanilla nerf model.
```bash
ns-train vanilla-nerf
```
Run a nerfacto model.
```bash
ns-train nerfacto
```
Run with nerfstudio data. You'll may have to change the ports, and be sure to forward the "websocket-port".
```
ns-train nerfacto --vis viewer --viewer.zmq-port 8001 --viewer.websocket-port 8002 nerfstudio-data --pipeline.datamanager.dataparser.data-directory data/nerfstudio/poster --pipeline.datamanager.dataparser.downscale-factor 4
```
## Visualizing training runs
If you using a fast NeRF variant (ie. Instant-NGP), we reccomend using our viewer. See our [viewer docs](viewer_quickstart.md) for more details. The viewer will allow interactive visualization of training in realtime.
Additionally, if you run everything with the default configuration, by default, we use [TensorBoard](https://www.tensorflow.org/tensorboard) to log all training curves, test images, and other stats. Once the job is launched, you will be able to track training by launching the tensorboard in `outputs/blender_lego/vanilla_nerf/<timestamp>/<events.tfevents>`.
```bash
tensorboard --logdir outputs
```
## Rendering a Trajectory
To evaluate the trained NeRF, we provide an evaluation script that allows you to do benchmarking (see our [benchmarking workflow](../developer_guides/benchmarking.md)) or to render out the scene with a custom trajectory and save the output to a video.
```bash
ns-eval render-trajectory --load-config=outputs/blender_lego/instant_ngp/2022-07-07_230905/config.yml --traj=spiral --output-path=output.mp4
```
Please note, this quickstart allows you to preform everything in a headless manner. We also provide a web-based viewer that allows you to easily monitor training or render out trajectories. See our [viewer docs](viewer_quickstart.md) for more.
## FAQ
- [TinyCUDA installation errors out with cuda mismatch](tiny-cuda-error)
(tiny-cuda-error)=
#### TinyCUDA installation errors out with cuda mismatch
While installing tiny-cuda, you run into: `The detected CUDA version mismatches the version that was used to compile PyTorch (10.2). Please make sure to use the same CUDA versions.`
**Solution**:
```
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html
```
- [Installation errors, File "setup.py" not found](pip-install-error)
(pip-install-error)=
#### Installation errors, File "setup.py" not found
When installing dependencies and nerfstudio with `pip install -e .`, you run into: `ERROR: File "setup.py" not found. Directory cannot be installed in editable mode`
**Solution**:
This can be fixed by upgrading pip to the latest version:
```
python -m pip install --upgrade pip
```
- [Runtime errors, "len(sources) > 0".](cuda-sources-error)
(cuda-sources-error)=
#### Runtime errors: "len(sources) > 0", "ctype = \_C.ContractionType(type.value) ; TypeError: 'NoneType' object is not callable".
When running `train.py `, an error occurs when installing cuda files in the backend code.
**Solution**:
This is a problem with not being able to detect the correct CUDA version, and can be fixed by updating the CUDA path environment variables:
```
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
export PATH=$PATH:$CUDA_HOME/bin
```
+56
View File
@@ -0,0 +1,56 @@
# Installation
### Create environment
We reccomend using conda to manage dependencies. Make sure to install [Conda](https://docs.conda.io/en/latest/miniconda.html) before preceding.
```bash
conda create --name nerfstudio -y python=3.8.13
conda activate nerfstudio
python -m pip install --upgrade pip
```
### Dependencies
Install pytorch with CUDA (this repo has been tested with CUDA 11.3) and [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn)
```bash
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
```
### Installing nerfstudio
Easy option:
```bash
pip install nerfstudio
```
If you would want the latest and greatest:
```bash
git clone git@github.com:plenoptix/nerfstudio.git
cd nerfstudio
pip install -e .
```
### Optional Installs
#### Tab completion (bash & zsh)
This needs to be rerun when the CLI changes, for example if nerfstudio is updated.
```bash
ns-install-cli
```
### Development packages
```bash
pip install -e.[dev]
pip install -e.[docs]
```
+1
View File
@@ -0,0 +1 @@
# Setting up Viewer