Rename to nerfstudio (#589)

This commit is contained in:
Matthew Tancik authored and GitHub committed 2022-09-29 23:42:15 -07:00
1 parent fae01d3ca6
commit 1f83ad03db
210 files changed
+3400 -3413

No files matched your search

+8 -8
View File
@@ -2,17 +2,17 @@
The tutorial below shows you how to host the viewer yourself and how you can use it to visualize training.
#### Connection between nerfactory and the Bridge Server
#### Connection between nerfstudio and the Bridge Server
In the center, we have the Bridge Server, which facilitates the connection between nerfactory code and the Client App. This server runs on the same machine that you are using nerfactory. It has a TCP Request/Reply (REQ/REP) connection that nerfactory can connect to with the Viewer object (left). We use [ZeroMQ](https://zeromq.org/), an open-sourced messaging library, to implement this lightweight TCP connection. The Viewer class can send commands to the Bridge Server and receive replies. The Bridge Server will either dispatch commands to the Client App via a websocket or it will return information stored in the Bridge Server state.
In the center, we have the Bridge Server, which facilitates the connection between nerfstudio code and the Client App. This server runs on the same machine that you are using nerfstudio. It has a TCP Request/Reply (REQ/REP) connection that nerfstudio can connect to with the Viewer object (left). We use [ZeroMQ](https://zeromq.org/), an open-sourced messaging library, to implement this lightweight TCP connection. The Viewer class can send commands to the Bridge Server and receive replies. The Bridge Server will either dispatch commands to the Client App via a websocket or it will return information stored in the Bridge Server state.
#### Connection between the Bridge Server and the Client App
The connection between the Bridge Server and the Client App works with WebSockets and WebRTC.
- **WebSocket connection** - The WebSocket is used by the Bridge Server to dispatch commands coming from the nerfactory TCP connection. Commands can be used for drawing primitives, for setting the transform of objects, for the setting various properties, and more.
- **WebSocket connection** - The WebSocket is used by the Bridge Server to dispatch commands coming from the nerfstudio TCP connection. Commands can be used for drawing primitives, for setting the transform of objects, for the setting various properties, and more.
- **WebRTC connection** - We use WebRTC to stream images being rendered from nerfactory. The websocket connection if first used to establish the WebRTC connection. Then, the Client App constantly publishes camera pose information to the Bridge Server and stores the camera information (intrinsics and extrinsics). This information is then queried from the nerfactory code, used to render an image with some Graph, and then the image is send over the TCP connection and dispatched via WebRTC to render the stream of images.
- **WebRTC connection** - We use WebRTC to stream images being rendered from nerfstudio. The websocket connection if first used to establish the WebRTC connection. Then, the Client App constantly publishes camera pose information to the Bridge Server and stores the camera information (intrinsics and extrinsics). This information is then queried from the nerfstudio code, used to render an image with some Graph, and then the image is send over the TCP connection and dispatched via WebRTC to render the stream of images.
## Getting started
@@ -21,7 +21,7 @@ The connection between the Bridge Server and the Client App works with WebSocket
The viewer server runs on the same machine that you use for training. The training code will connect to the server with a lightweight TCP connection.
```
cd nerfactory
cd nerfstudio
# run the server on your machine
# this will run in the background
@@ -36,7 +36,7 @@ It should print out something of the form:
> We will host the viewer online in the future, but for now we have to run it locally.
```shell
cd nerfactory/viewer/app
cd nerfstudio/viewer/app
```
Install npm (to install yarn) and yarn
@@ -75,7 +75,7 @@ PORT=4000
ESLINT_NO_DEV_ERRORS=true
```
#### Running the nerfactory Code
#### Running the nerfstudio Code
You can now simply run a training job and visualize progress by enabling the viewer during training:
@@ -83,7 +83,7 @@ You can now simply run a training job and visualize progress by enabling the vie
python scripts/train.py --config-name=graph_instant_ngp.yaml viewer.enable=true
```
- **Notebook demo** - See [Programming the viewer](viewer_notebook.ipynb) for an overview for how to interact with the viewer with the Viewer object from nerfactory.
- **Notebook demo** - See [Programming the viewer](viewer_notebook.ipynb) for an overview for how to interact with the viewer with the Viewer object from nerfstudio.
## Acknowledgements and references
+3 -3
View File
@@ -38,8 +38,8 @@
"import numpy as np\n",
"import torch\n",
"import mediapy as media\n",
"from nerfactory.data.structs import SceneBounds\n",
"from nerfactory.cameras.cameras import PinholeCamera\n",
"from nerfstudio.data.structs import SceneBounds\n",
"from nerfstudio.cameras.cameras import PinholeCamera\n",
"import time"
]
},
@@ -122,7 +122,7 @@
"outputs": [],
"source": [
"# connect to the viewer from Python\n",
"from nerfactory.viewer.server.visualizer import Viewer\n",
"from nerfstudio.viewer.server.visualizer import Viewer\n",
"\n",
"vis = Viewer(zmq_url=\"tcp://127.0.0.1:6000\")"
]
+1 -1
View File
@@ -15,7 +15,7 @@ The viewer is built using [ThreeJS](https://threejs.org/) and packaged into a [R
python scripts/train.py --config-name=graph_instant_ngp.yaml viewer.enable=True
```
This pulls up a hosted web-viewer which is the fastest way to get started with visualizing your nerfactory training progress!
This pulls up a hosted web-viewer which is the fastest way to get started with visualizing your nerfstudio training progress!
![viewer](imgs/bulldozer.png)