rename files and add project scripts (#566)

* scripts

* b

* fix file
This commit is contained in:
Ethan Weber authored and GitHub committed 2022-09-27 15:02:55 -07:00
1 parent 8af3a7a385
commit cf34ef4bd0
26 files changed
+384 -355

No files matched your search

+3 -9
View File
@@ -2,12 +2,10 @@
The tutorial below shows you how to host the viewer yourself and how you can use it to visualize training.
#### Connection between nerfactory and the Bridge Server
In the center, we have the Bridge Server, which facilitates the connection between nerfactory code and the Client App. This server runs on the same machine that you are using nerfactory. It has a TCP Request/Reply (REQ/REP) connection that nerfactory can connect to with the Viewer object (left). We use [ZeroMQ](https://zeromq.org/), an open-sourced messaging library, to implement this lightweight TCP connection. The Viewer class can send commands to the Bridge Server and receive replies. The Bridge Server will either dispatch commands to the Client App via a websocket or it will return information stored in the Bridge Server state.
#### Connection between the Bridge Server and the Client App
The connection between the Bridge Server and the Client App works with WebSockets and WebRTC.
@@ -16,10 +14,8 @@ The connection between the Bridge Server and the Client App works with WebSocket
- **WebRTC connection** - We use WebRTC to stream images being rendered from nerfactory. The websocket connection if first used to establish the WebRTC connection. Then, the Client App constantly publishes camera pose information to the Bridge Server and stores the camera information (intrinsics and extrinsics). This information is then queried from the nerfactory code, used to render an image with some Graph, and then the image is send over the TCP connection and dispatched via WebRTC to render the stream of images.
## Getting started
#### Running the Bridge Server
The viewer server runs on the same machine that you use for training. The training code will connect to the server with a lightweight TCP connection.
@@ -35,7 +31,6 @@ It should print out something of the form:
"ZMQWebSocketBridge using zmq_url=tcp://127.0.0.1:6000 and websocket_port=8051"
```
#### Running the Client App
> We will host the viewer online in the future, but for now we have to run it locally.
@@ -67,6 +62,7 @@ yarn start
```
Forward port and open in your local browser. The URL takes the following form, `http://localhost:<forwarded_react_port>?localhost:<forwarded_backend_tcp_port>`. Notice the "?" character.
- http://localhost:4000?localhost:8051
Possibly edit the port in `app/.env.development` file with the following to adjust the PORT, for example.
@@ -79,18 +75,16 @@ PORT=4000
ESLINT_NO_DEV_ERRORS=true
```
#### Running the nerfactory Code
You can now simply run a training job and visualize progress by enabling the viewer during training:
```bash
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=true
python scripts/train.py --config-name=graph_instant_ngp.yaml viewer.enable=true
```
- **Notebook demo** - See [Programming the viewer](viewer_notebook.ipynb) for an overview for how to interact with the viewer with the Viewer object from nerfactory.
## Acknowledgements and references
We thank [Robin Deits](https://github.com/rdeits) and other contributors to the following repos, which we've started with and modified and extended for our use.
@@ -106,4 +100,4 @@ Here are other resources that we've used and/or have found helpful while buildin
```
```
```
+1 -1
View File
@@ -12,7 +12,7 @@ The viewer is built using [ThreeJS](https://threejs.org/) and packaged into a [R
2. Run training script with viewer enabled. Note that you should only use the viewer with a fast pipeline, such as instant ngp.
```shell
python scripts/run_train.py --config-name=graph_instant_ngp.yaml viewer.enable=True
python scripts/train.py --config-name=graph_instant_ngp.yaml viewer.enable=True
```
This pulls up a hosted web-viewer which is the fastest way to get started with visualizing your nerfactory training progress!