doc refactoring (#267)

* reformatted docs; moved visualizer section; added references to top; hiding some doctrees etc.

* changed realtime viewer tab
This commit is contained in:
evonneng authored and GitHub committed 2022-08-10 16:15:44 -07:00
1 parent 841e03a9a3
commit 3cb44f3a8e
25 files changed
+884 -761

No files matched your search

+5 -5
View File
@@ -18,11 +18,11 @@ Neural Volumetric Rendering
All-in-one repository for state-of-the-art NeRFs.
nerfactory provides a simple API that allows for a seamless and simplified end-to-end process of creating, training, and testing NeRFs.
Nerfactory provides a simple API that allows for a seamless and simplified end-to-end process of creating, training, and testing NeRFs.
The library supports a more interpretable implementation of NeRFs by modularizing each component.
With more modular NeRFs, not only does your code become far more user-friendly, but using this framework also makes it easier for the community to build upon your implementation.
It's as simple as plug and play with nerfactory!
It's as simple as plug and play with Nerfactory!
If you have further questions or want a feature that is not yet supported, please do not hesitate to reach out to the Plenoptix team.
@@ -37,9 +37,9 @@ Contents
:caption: Getting Started
Quickstart<tutorials/quickstart_index>
tutorials/data_index
tutorials/creating_graphs.md
tutorials/system_overview.md
tutorials/data/data_index
tutorials/models/creating_pipelines
tutorials/viewer/using_viewer
.. toctree::
:hidden:
-1
View File
@@ -9,5 +9,4 @@ It can be difficult getting started with NeRFs. The field is still quite new and
Ray Samplers<visualize_samplers.ipynb>
Spatial Distortions<visualize_spatial_distortions.ipynb>
Encoders<visualize_encoders.ipynb>
Viewer<visualize_viewer.ipynb>
```
+25 -23
View File
@@ -38,6 +38,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(pinhole)=\n",
"## Pinhole Camera Model\n",
"\n",
"The simplest and most used camera model. Most standard cameras can be sufficiently approximated using the pinhole camera model.\n",
@@ -13868,6 +13869,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(equirectangular)=\n",
"## Equirectangular (360°) Camera Models\n",
"\n",
"An Equirectangular camera images the scene in all directions at the same time.\n",
@@ -16921,7 +16923,7 @@
0,
0.03252455219626427,
0,
-6.837967969630654e-09,
-6.837967969630654e-9,
0,
-0.03252456337213516,
0,
@@ -16951,7 +16953,7 @@
0,
-0.03252459317445755,
0,
1.865462451178246e-09,
1.865462451178246e-9,
0,
0.03252459689974785,
0,
@@ -16981,7 +16983,7 @@
0,
0.09438992291688919,
0,
-1.9844557996862022e-08,
-1.9844557996862022e-8,
0,
-0.09438996762037277,
0,
@@ -17011,7 +17013,7 @@
0,
-0.09439004957675934,
0,
5.41378319951491e-09,
5.41378319951491e-9,
0,
0.09439005702733994,
0,
@@ -17041,7 +17043,7 @@
0,
0.1470157355070114,
0,
-3.0908619663705394e-08,
-3.0908619663705394e-8,
0,
-0.1470157951116562,
0,
@@ -17071,7 +17073,7 @@
0,
-0.14701592922210693,
0,
8.432163411953297e-09,
8.432163411953297e-9,
0,
0.14701594412326813,
0,
@@ -17101,7 +17103,7 @@
0,
0.1852506399154663,
0,
-3.894713174190656e-08,
-3.894713174190656e-8,
0,
-0.18525072932243347,
0,
@@ -17131,7 +17133,7 @@
0,
-0.1852508783340454,
0,
1.0625146096288063e-08,
1.0625146096288063e-8,
0,
0.1852508932352066,
0,
@@ -17161,7 +17163,7 @@
0,
0.20535191893577576,
0,
-4.3173230324100587e-08,
-4.3173230324100587e-8,
0,
-0.20535200834274292,
0,
@@ -17191,7 +17193,7 @@
0,
-0.20535218715667725,
0,
1.1778065633905044e-08,
1.1778065633905044e-8,
0,
0.20535220205783844,
0,
@@ -17221,7 +17223,7 @@
0,
0.20535190403461456,
0,
-4.317322677138691e-08,
-4.317322677138691e-8,
0,
-0.20535199344158173,
0,
@@ -17251,7 +17253,7 @@
0,
-0.20535217225551605,
0,
1.1778064745726624e-08,
1.1778064745726624e-8,
0,
0.20535218715667725,
0,
@@ -17281,7 +17283,7 @@
0,
0.1852506548166275,
0,
-3.894713529462024e-08,
-3.894713529462024e-8,
0,
-0.18525072932243347,
0,
@@ -17311,7 +17313,7 @@
0,
-0.1852508932352066,
0,
1.0625146984466483e-08,
1.0625146984466483e-8,
0,
0.1852509081363678,
0,
@@ -17341,7 +17343,7 @@
0,
0.1470157355070114,
0,
-3.0908619663705394e-08,
-3.0908619663705394e-8,
0,
-0.1470157951116562,
0,
@@ -17371,7 +17373,7 @@
0,
-0.14701592922210693,
0,
8.432163411953297e-09,
8.432163411953297e-9,
0,
0.14701594412326813,
0,
@@ -17401,7 +17403,7 @@
0,
0.0943898856639862,
0,
-1.9844549115077825e-08,
-1.9844549115077825e-8,
0,
-0.09438993036746979,
0,
@@ -17431,7 +17433,7 @@
0,
-0.09439001232385635,
0,
5.413780979068861e-09,
5.413780979068861e-9,
0,
0.09439001977443695,
0,
@@ -17461,7 +17463,7 @@
0,
0.03252454474568367,
0,
-6.837966637363024e-09,
-6.837966637363024e-9,
0,
-0.032524559646844864,
0,
@@ -17491,7 +17493,7 @@
0,
-0.03252458572387695,
0,
1.865462007089036e-09,
1.865462007089036e-9,
0,
0.03252458944916725,
0,
@@ -19606,7 +19608,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.8.12 ('nerfactory')",
"display_name": "Python 3.8.13 ('nerfactory')",
"language": "python",
"name": "python3"
},
@@ -19620,11 +19622,11 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.12"
"version": "3.8.13"
},
"vscode": {
"interpreter": {
"hash": "34c28001ff35fb390494047002768a8182dcf55b1b11415165e62ea61557ab83"
"hash": "100252972d4ee5286ab455ef1c778abde05b4affca9d1be78633d3797abf7e8c"
}
}
},
+12 -6
View File
@@ -4,13 +4,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Field Encoders\n"
"# Field Encoders"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(nerf)=\n",
"## NeRF Positional Encoding\n",
"First introduced in the original NeRF paper. This encoding assumes the inputs are between zero and one and can opperate on any dimensional input."
]
@@ -164,6 +165,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(rff)=\n",
"## Random Fourier Feature (RFF) Encoding\n",
"This encoding assumes the inputs are between zero and one and can opperate on any dimensional input."
]
@@ -309,6 +311,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(hash)=\n",
"## Hash Encoding\n",
"The hash incoding was originally introduced in Instant-NGP. The encoding is optimized during training. This is a visualization of the initialization."
]
@@ -410,6 +413,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(spherical)=\n",
"## Spherical Harmonic Encoding\n",
"Encode direction using spherical harmoincs. (Mostly used to encode viewing direction)"
]
@@ -529,11 +533,8 @@
}
],
"metadata": {
"interpreter": {
"hash": "024fe9dea615e4e34961116ff517c7e9f5910440e3d2c8eb4560299f11aaf2e7"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": "Python 3.8.13 ('nerfactory')",
"language": "python",
"name": "python3"
},
@@ -547,7 +548,12 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
"version": "3.8.13"
},
"vscode": {
"interpreter": {
"hash": "100252972d4ee5286ab455ef1c778abde05b4affca9d1be78633d3797abf7e8c"
}
}
},
"nbformat": 4,
+6 -3
View File
@@ -37,6 +37,7 @@
"Nerfactory calculates $pixel area$ as $dx * dy$ where $dx$ and $dy$ are the distance between points neighboring rays in the $x$ and $y$ direction a distance of 1 from the origin.\n",
":::\n",
"\n",
"(3d)=\n",
"## 3D Frustum\n"
]
},
@@ -7631,6 +7632,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(frustum_as_points)=\n",
"## Representing Frustums as Points\n",
"\n",
"These frustums can be converted into point samples using `frustum.get_positions()`.\n"
@@ -8816,6 +8818,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(frustum_as_gaussian)=\n",
"## Representing Frustums as Gaussians\n",
"\n",
"We can also appoximate the frustums using Gaussians with `frustum.get_gaussian_blob()`.\n"
@@ -68226,7 +68229,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.8.12 ('nerfactory')",
"display_name": "Python 3.8.13 ('nerfactory')",
"language": "python",
"name": "python3"
},
@@ -68240,11 +68243,11 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.12"
"version": "3.8.13"
},
"vscode": {
"interpreter": {
"hash": "41f62990c6e03ccb48eec29860b83c64e20fcd45204ae5014afab6809cd6f1f1"
"hash": "100252972d4ee5286ab455ef1c778abde05b4affca9d1be78633d3797abf7e8c"
}
}
},
File diff suppressed because it is too large. Load diff
+13 -12
View File
@@ -1,8 +1,10 @@
# Benchmarking workflow
We make it easy to benchmark your new NeRF against the standard Blender dataset.
#### Launching training on Blender dataset
We make it easy to benchmark your new NeRF against the standard Blender dataset.
(train)=
## Launching training on Blender dataset
To start, you will need to train your NeRF on each of the blender objects.
To launch training jobs automatically on each of these items, you can call:
@@ -27,8 +29,8 @@ A full example would be:
The script will automatically launch training on all of the items and save the checkpoints in an output directory with the current date and timestamp.
#### Benchmarking on Blender
(eval)=
## Evaluating trained Blender models
Once you have launched training, and training converges, you can test your method with `scripts/benchmarking/launch_eval_blender.sh`.
Say we ran a benchmark on 08-10-2022 for instant_ngp. By default, the train script will save the benchmarks in the following format:
@@ -47,20 +49,19 @@ outputs
...
```
In the above example, we can break down the path into component flags that will be passed into the benchmarking script:
If we wanted to run the benchmark on all the blender data for the above example, we would run:
```bash
./scripts/benchmarking/launch_eval_blender.sh -c graph_instant_ngp -o ./outputs/ -m 08 -d 10 -y 2022 -s 172517 -g 4 5 6 7
```
The flags used in the benchmarking script are defined as follows:
* `-c`: config name (e.g. `graph_instant_ngp`). This should be the same as what was passed in for -c in the train script.
* `-o`: base output directory for where all of the benchmarks are stored (e.g. `outputs/`). Corresponds to the hydra base dir.
* `-m`: month of benchmark (e.g. `08`) of format 02d.
* `-d`: date of benchmark (e.g. `10`) of format 02d.
* `-y`: year of benchmark (e.g. `2022`).
* `-s`: seconds timestamp of benchmark (e.g. `172517`) of format 06d.
If we wanted to run the full benchmark for all the blender data given the above example, we would run the following:
```bash
./scripts/benchmarking/launch_eval_blender.sh -c graph_instant_ngp -o ./outputs/ -m 08 -d 10 -y 2022 -s 172517 -g 4 5 6 7
```
where `-g` again specifies the gpus to use and if not specified (no -g flag), will automaticaly search for available gpus.
* `-g`: specifies the gpus to use and if not specified (no -g flag), will automaticaly search for available gpus.
The script will simultaneously run the benchmarking across all the objects in the blender dataset and calculates the PSNR/FPS/other stats. The results are saved as .json files in the `-o` directory with the following format:
+3 -5
View File
@@ -1,10 +1,8 @@
.. _tooling:
.. toctree::
:maxdepth: 1
hydra.md
logging.md
viewer.md
profiling.md
logging_profiling.md
benchmarking.md
-30
View File
@@ -1,30 +0,0 @@
# Logging Features
We provide integration with multiple logging interfaces to log images and statistics during training.
All of these loggers are configurable via the config:
```yaml
# e.g. configs/graphs_default.yaml
logging:
steps_per_log: 10 # how often to log losses/images
max_buffer_size: 20 # defines how many steps to average over (if reporting averages)
writer: # definition of all writers you want to use
TensorboardWriter:
log_dir: "./" # must specify the output path for all writers
LocalWriter:
log_dir: "./"
stats_to_track:
[ITER_LOAD_TIME, ITER_TRAIN_TIME, RAYS_PER_SEC, CURR_TEST_PSNR]
max_log_size: 10
```
Currently, we provide support for the following loggers:
1. [Tensorboard](https://www.tensorflow.org/tensorboard): `TensorBoardWriter`
2. [Weights and Biases](https://wandb.ai/site): `WandbWriter`
3. Local writer: `LocalWriter`
The `LocalWriter` simply outputs to the terminal. However, you can specify additional parameters to customize the logging.
* stats_to_track: all the stats that you want to print to the terminal (see list under `EventName` in `utils/writer.py`). You can add or remove any of the defined enums.
* max_log_size: how much content to print onto the screen (e.g. only print 10 lines on the screen at a time). If 0, will print everything without deleting any previous lines.
@@ -1,8 +1,65 @@
# Profiling and debugging
# Loggers and Profilers
## Profilers
(logging_support)=
## Logging support
We provide integration with multiple logging interfaces to log images and statistics during training.
All of these loggers are configurable via the config:
```yaml
# e.g. configs/graphs_default.yaml
logging:
steps_per_log: 10 # how often to log losses/images
max_buffer_size: 20 # defines how many steps to average over (if reporting averages)
writer: # definition of all writers you want to use
TensorboardWriter:
log_dir: "./" # must specify the output path for all writers
LocalWriter:
log_dir: "./"
stats_to_track:
[ITER_LOAD_TIME, ITER_TRAIN_TIME, RAYS_PER_SEC, CURR_TEST_PSNR]
max_log_size: 10
```
Currently, we provide support for the following loggers, which can be added to the `logging.writer` section of the config.
(tensorboard)=
#### Tensorboard
To log training stats and images with [tensorboard](https://www.tensorflow.org/tensorboard):
```
TensorboardWriter:
log_dir: "./" # the output path for which the event files are written
```
(wnb)=
#### Weights and Biases
To log training stats and images with [wandb](https://wandb.ai/site):
```
WandbWriter:
log_dir: "./" # the output path for which the run files are written
```
(local)=
#### Local writer
```
LocalWriter:
log_dir: "./" # if images are saved, where the images would be saved
stats_to_track:
[ITER_LOAD_TIME, ITER_TRAIN_TIME, RAYS_PER_SEC, CURR_TEST_PSNR]
max_log_size: 10
```
The `LocalWriter` simply outputs to the terminal. However, you can specify additional parameters to customize the logging.
* `stats_to_track`: all the stats that you want to print to the terminal (see list under `EventName` in `utils/writer.py`). You can add or remove any of the defined enums.
* `max_log_size`: how much content to print onto the screen (e.g. only print 10 lines on the screen at a time). If 0, will print everything without deleting any previous lines.
(code_profiler)=
## Code profiling support
We provide built-in performance profiling capabilities to make it easier for you to debug and assess the performance of your code.
(inhouse)=
#### In-house profiler
You can use our built-in profiler by enabling the profiler in the config:
@@ -33,6 +90,7 @@ At termination of training or end of the training run, the profiler will print o
Use this profiler if there are specific functions that you want to measure the times for.
(pyspy)=
#### Profiling with PySpy
If you want to profile the entire codebase, we provide functionality to generate flame graphs or get real-time reports using [PySpy](https://github.com/benfred/py-spy).
@@ -58,6 +116,4 @@ In both cases, `<python_command>` can be replaced with any command you would nor
```bash
./scripts/debugging/profile.sh -t flame -o flame.svg -p scripts/run_train.py data/dataset=blender_lego
```
## Debugging
```
+11
View File
@@ -0,0 +1,11 @@
Data Setup
----------
# TODO: high level summary
.. toctree::
:maxdepth: 1
choosing_dataset.md
creating_dataset.md
custom_dataset_format.md
-9
View File
@@ -1,9 +0,0 @@
Data Setup
-----------
.. toctree::
:maxdepth: 1
data/choosing_dataset.md
data/creating_dataset.md
data/custom_dataset_format.md
@@ -148,4 +148,4 @@ python scripts/run_train.py --config-name vanilla_nerf
And now you have a brand-new NeRF in training! For testing and visualizing, simply refer to steps 4-5 in the [quickstart guide](https://github.com/plenoptix/nerfactory#quickstart).
To help you get started, we also provide additional training tools such as profiling, logging, and debugging. Please refer to our [features guide](../tooling/index.rst).
To help you get started, we also provide additional training tools such as profiling, logging, and debugging. Please refer to our [features guide](../../tooling/index.rst).
@@ -0,0 +1,10 @@
Creating custom pipelines
-------------------------
# TODO: high level summary
.. toctree::
:maxdepth: 1
system_overview.md
creating_graphs.md
+4
View File
@@ -0,0 +1,4 @@
# System Overview
TODO: something with a higher-level node-graph image for wiring together the models
TODO: maybe also description of what the pipeline is constructed of
+19 -1
View File
@@ -88,7 +88,7 @@ Run a faster version with instant ngp using tcnn (without the viewer).
python scripts/run_train.py --config-name=graph_instant_ngp.yaml
```
Run with the viewer. However, you'll have to start the viewer server first. (See the docs.)
Run with the viewer. However, you'll have to start the viewer server first. (See [viewer docs](../tutorials/viewer/viewer_quickstart.md))
```bash
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=true
@@ -113,6 +113,24 @@ bash environment/run_tensorboard.sh
# Rendering a Trajectory
To evaluate the trained NeRF, we provide an evaluation script that allows you to do benchmarking (see our [benchmarking workflow](../tooling/benchmarking.md)) or to render out the scene with a custom trajectory and save the output to a video.
```bash
python scripts/run_eval.py --method=traj --traj=spiral --output-filename=output.mp4 --config-name=graph_instant_ngp.yaml trainer.resume_train.load_dir=outputs/blender_lego/instant_ngp/2022-07-07_230905/checkpoints
```
Please note, this quickstart allows you to preform everything in a headless manner. We also provide a web-based visualizer that allows you to easily monitor training or render out trajectories. See our [viewer docs](../tutorials/viewer/viewer_quickstart.md) for more.
# FAQ
* [TinyCUDA installation errors out with cuda mismatch](tiny-cuda-error)
(tiny-cuda-error)=
#### TinyCUDA installation errors out with cuda mismatch
While installing tiny-cuda, you run into: `The detected CUDA version mismatches the version that was used to compile PyTorch (10.2). Please make sure to use the same CUDA versions.`
**Solution**:
```
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html
```
+5 -2
View File
@@ -24,9 +24,12 @@ The library supports a more interpretable implementation of NeRFs by modularizin
With more modular NeRFs, not only does your code become far more user-friendly, but using this framework also makes it easier for the community to build upon your implementation.
# TODO: high level summary of what is in the quickstart tabs.
Quickstart
-----------
.. toctree::
quickstart.md
:maxdepth: 1
quickstart.md
-1
View File
@@ -1 +0,0 @@
# System Overview
+21
View File
@@ -0,0 +1,21 @@
# FAQ
* [Engine node incompatible](engine-error)
(engine-error)=
#### Engine node incompatible
While running `yarn install`, you run into: `The engine "node" is incompatible with this module.`
**Solution**:
Install nvm with instructions at [instructions](https://heynode.com/tutorial/install-nodejs-locally-nvm/).
```shell
nvm install 17.8.0
```
If you cannot install nvm, try ignoring the engines
```
yarn install --ignore-engines
```
@@ -1,30 +1,13 @@
# Real-time Viewer
# For developers
We provide a real-time viewer that can be used during training or to view a trained model. The purpose of the viewer is to be able to freely move the camera and see the reconstruction from novel viewpoints. It can also be used to draw various primitives (e.g., bounding boxes, camera frustums, etc.).
The viewer is built using [ThreeJS](https://threejs.org/) and packaged into a [ReactJS](https://reactjs.org/) application. This client viewer application will connect via a websocket to a server running on your machine. The following figure helps to illustrate how our viewer framework works:
![visualize_dataset](media/viewer_figure.png)
## Quick start
WOP: This sections is not ready. Please proceed to the "For developers" section below.
1. Open the Client App Viewer at ... `TODO(ethan): host the static website`.
2. Run training script with viewer enabled. Note that you should only use the viewer with a fast pipeline, such as instant ngp.
```shell
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=True
```
<hr>
## For developers
The tutorial below shows you how to host the viewer yourself and how you can use it to visualize training.
(nerfactory_bridge)=
#### Connection between nerfactory and the Bridge Server
In the center, we have the Bridge Server, which facilitates the connection between nerfactory code and the Client App. This server runs on the same machine that you are using nerfactory. It has a TCP Request/Reply (REQ/REP) connection that nerfactory can connect to with the Viewer object (left). We use [ZeroMQ](https://zeromq.org/), an open-sourced messaging library, to implement this lightweight TCP connection. The Viewer class can send commands to the Bridge Server and receive replies. The Bridge Server will either dispatch commands to the Client App via a websocket or it will return information stored in the Bridge Server state.
(bridge_client)=
#### Connection between the Bridge Server and the Client App
The connection between the Bridge Server and the Client App works with WebSockets and WebRTC.
@@ -33,8 +16,10 @@ The connection between the Bridge Server and the Client App works with WebSocket
- **WebRTC connection** - We use WebRTC to stream images being rendered from nerfactory. The websocket connection if first used to establish the WebRTC connection. Then, the Client App constantly publishes camera pose information to the Bridge Server and stores the camera information (intrinsics and extrinsics). This information is then queried from the nerfactory code, used to render an image with some Graph, and then the image is send over the TCP connection and dispatched via WebRTC to render the stream of images.
(getting_started)=
## Getting started
(bridge)=
#### Running the Bridge Server
The viewer server runs on the same machine that you use for training. The training code will connect to the server with a lightweight TCP connection.
@@ -50,6 +35,7 @@ It should print out something of the form:
"ZMQWebSocketBridge using zmq_url=tcp://127.0.0.1:6000 and websocket_port=8051"
```
(client)=
#### Running the Client App
> We will host the viewer online in the future, but for now we have to run it locally.
@@ -66,7 +52,7 @@ npm install --global yarn
```
Install nvm and set the node version
Install nvm with instructions at e.g., https://heynode.com/tutorial/install-nodejs-locally-nvm/
Install nvm with [instructions](https://heynode.com/tutorial/install-nodejs-locally-nvm/).
```shell
nvm install 17.8.0
@@ -93,10 +79,18 @@ PORT=4000
ESLINT_NO_DEV_ERRORS=true
```
(nerfactory)=
#### Running the nerfactory Code
- **Notebook demo** - See `docs/notebooks/visualize_viewer.ipynb` for an overview for how to interact with the viewer with the Viewer object from nerfactory.
You can now simply run a training job and visualize progress by enabling the viewer during training:
```bash
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=true
```
- **Notebook demo** - See [Programming the viewer](viewer_notebook.ipynb) for an overview for how to interact with the viewer with the Viewer object from nerfactory.
(ack)=
## Acknowledgements and references
We thank [Robin Deits](https://github.com/rdeits) and other contributors to the following repos, which we've started with and modified and extended for our use.
@@ -112,4 +106,4 @@ Here are other resources that we've used and/or have found helpful while buildin
```
```
```
File renamed without changes.

Before

Width:  |  Height:  |  Size: 286 KiB

After

Width:  |  Height:  |  Size: 286 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 218 KiB

After

Width:  |  Height:  |  Size: 218 KiB

+12
View File
@@ -0,0 +1,12 @@
Real-time web viewer
====================
# TODO: high level summary
.. toctree::
:maxdepth: 1
viewer_quickstart.md
for_developers.md
viewer_notebook.ipynb
faq.md
@@ -5,7 +5,7 @@
"id": "3c00934d-0f14-4c1a-a563-94d627b377e5",
"metadata": {},
"source": [
"# Visualize with our Viewer\n",
"# Programming the viewer\n",
"\n",
"The real-time web viewer enables communication between Python and the ReactJS Client Viewer Application. Here we provide a walkthrough for how to use the viewer to draw information as well as read information from the GUI.\n"
]
@@ -512,7 +512,7 @@
},
"vscode": {
"interpreter": {
"hash": "e691f033e0f2f1b9c0a11e7e81375a1e27aec87a71fd1b1eaa545f7a5e61b3f6"
"hash": "100252972d4ee5286ab455ef1c778abde05b4affca9d1be78633d3797abf7e8c"
}
}
},
@@ -0,0 +1,22 @@
# Quickstart
We provide a real-time, web-based viewer that can be used during training or to view a trained model. The purpose of the viewer is to be able to freely move the camera and see the reconstruction from novel viewpoints. It can also be used to draw various primitives (e.g., bounding boxes, camera frustums, etc.).
The viewer is built using [ThreeJS](https://threejs.org/) and packaged into a [ReactJS](https://reactjs.org/) application. This client viewer application will connect via a websocket to a server running on your machine. The following figure helps to illustrate how our viewer framework works:
![visualize_dataset](imgs/viewer_figure.png)
## Run hosted web viewer
[WOP: This sections is not ready. Please proceed to the "For developers" section.]
1. Open the Client App Viewer at ... `TODO(ethan): host the static website`.
2. Run training script with viewer enabled. Note that you should only use the viewer with a fast pipeline, such as instant ngp.
```shell
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=True
```
This pulls up a hosted web-viewer which is the fastest way to get started with visualizing your nerfactory training progress!
If you want to host the viewer locally/yourself, please proceed to "For devleopers" section.