mirror of
https://github.com/nerfstudio-project/nerfstudio.git
synced 2026-11-14 11:54:01 +08:00
Documenting viewer and faster image sending (#86)
* updated three.js version. only send camera information when it moves * changing code back to updating camera constantly because packets seem to be lost * name change * starting viewer docs * new workflow * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * ga * homepage * homepage * / to ? * more documentation * fixed none error * instant ngp fix * faster image sending over tcp. instant ngp bin fix. lowest res when moving * black * requirements
This commit is contained in:
Ethan Weber authored and 
GitHub committed
1 parent
d0f0f8da2d
commit
44c5e3264c
28 files changed
+28943
-609
No files matched your search
@@ -0,0 +1,57 @@
|
||||
name: Build and deploy viewer client app.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
pull_request:
|
||||
branches: [master]
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./pyrad/viewer/app
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@master
|
||||
|
||||
- name: Install Node.js
|
||||
uses: actions/setup-node@v1
|
||||
with:
|
||||
node-version: 17.8.0
|
||||
|
||||
- name: Install packages
|
||||
run: npm install
|
||||
|
||||
- name: Build project
|
||||
run: CI=false npm run build
|
||||
|
||||
# - name: Run tests
|
||||
# run: npm test
|
||||
|
||||
- name: Upload production-ready build files
|
||||
uses: actions/upload-artifact@v2
|
||||
with:
|
||||
name: production-files
|
||||
path: ./pyrad/viewer/app/build
|
||||
|
||||
deploy:
|
||||
name: Deploy
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
# if: github.ref == 'refs/heads/master'
|
||||
|
||||
steps:
|
||||
- name: Download artifact
|
||||
uses: actions/download-artifact@v2
|
||||
with:
|
||||
name: production-files
|
||||
path: ./pyrad/viewer/app/build
|
||||
|
||||
- name: Deploy to gh-pages
|
||||
uses: peaceiris/actions-gh-pages@v3
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: ./pyrad/viewer/app/build
|
||||
@@ -28,6 +28,7 @@ Contents
|
||||
tutorials/data_setup.rst
|
||||
tutorials/creating_graphs.md
|
||||
Contributing <tutorials/contributing.md>
|
||||
tutorials/viewer.md
|
||||
|
||||
.. toctree::
|
||||
:hidden:
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 218 KiB |
@@ -0,0 +1,100 @@
|
||||
# Real-time Viewer
|
||||
|
||||
We provide a real-time viewer that can be used during training or to view a trained model. The purpose of the viewer is to be able to freely move the camera and see the reconstruction from novel viewpoints. It can also be used to draw various primitives (e.g., bounding boxes, camera frustums, etc.).
|
||||
|
||||
The viewer is built using [ThreeJS](https://threejs.org/) and packaged into a [ReactJS](https://reactjs.org/) application. This client viewer application will connect via a websocket to a server running on your machine. The following figure helps to illustrate how our viewer framework works:
|
||||
|
||||
|
||||
|
||||
## Quick start
|
||||
|
||||
```
|
||||
# run training script with the viewer
|
||||
|
||||
```
|
||||
|
||||
<hr>
|
||||
|
||||
## For developers
|
||||
|
||||
#### Connection between PyRad and the Bridge Server
|
||||
|
||||
In the center, we have the Bridge Server, which facilitates the connection between PyRad code and the Client App. This server runs on the same machine that you are using PyRad. It has a TCP Request/Reply (REQ/REP) connection that PyRad can connect to with the Viewer object (left). We use [ZeroMQ](https://zeromq.org/), an open-sourced messaging library, to implement this lightweight TCP connection. The Viewer class can send commands to the Bridge Server and receive replies. The Bridge Server will either dispatch commands to the Client App via a websocket or it will return information stored in the Bridge Server state.
|
||||
|
||||
#### Connection between the Bridge Server and the Client App
|
||||
|
||||
The connection between the Bridge Server and the Client App works with WebSockets and WebRTC.
|
||||
|
||||
- **WebSocket connection** - The WebSocket is used by the Bridge Server to dispatch commands coming from the PyRad TCP connection. Commands can be used for drawing primitives in the [JSON Object Scene format](https://github.com/mrdoob/three.js/wiki/JSON-Object-Scene-format-4), for setting the transform of objects, for the setting various properties, and more.
|
||||
|
||||
- **WebRTC connection** - We use WebRTC to stream images being rendered from PyRad. The websocket connection if first used to establish the WebRTC connection. Then, the Client App constantly publishes camera pose information to the Bridge Server and stores the camera information (intrinsics and extrinsics). This information is then queried from the PyRad code, used to render an image with some Graph, and then the image is send over the TCP connection and dispatched via WebRTC to render the stream of images.
|
||||
|
||||
## Getting started
|
||||
|
||||
#### Running the Bridge Server
|
||||
|
||||
The viewer server runs on the same machine that you use for training. The training code will connect to the server with a lightweight TCP connection.
|
||||
|
||||
```
|
||||
cd pyrad
|
||||
|
||||
# run the server on your machine
|
||||
# this will run in the background
|
||||
python scripts/run_viewer_server.py
|
||||
|
||||
It should print out something of the form:
|
||||
"ZMQWebSocketBridge using zmq_url=tcp://127.0.0.1:6000 and websocket_port=8051"
|
||||
```
|
||||
|
||||
#### Running the Client App
|
||||
|
||||
> We will host the viewer online in the future, but for now we have to run it locally.
|
||||
|
||||
```
|
||||
cd pyard
|
||||
|
||||
cd pyrad/viewer/app
|
||||
|
||||
# install yarn and package.json dependencies
|
||||
npm install --global yarn
|
||||
|
||||
yarn install
|
||||
yarn start
|
||||
|
||||
# forward port and open in your local browser
|
||||
# the URL takes the following form
|
||||
# http://localhost:<forwarded_react_port>/localhost:<forwarded_backend_tcp_port>
|
||||
http://localhost:4000/localhost:8051
|
||||
```
|
||||
|
||||
Possibly edit the port in `app/.env.development` file with the following:
|
||||
|
||||
```
|
||||
BROWSER=none
|
||||
FAST_REFRESH=false
|
||||
HOST=localhost
|
||||
PORT=4000
|
||||
```
|
||||
|
||||
#### Running the PyRad Code
|
||||
|
||||
- **Notebook demo** - See `notebooks/visualize_viewer.ipynb` for an overview for how to interact with the viewer with the Viewer object from PyRad.
|
||||
|
||||
## Features checklist
|
||||
|
||||
- [x] WebRTC video stream
|
||||
- [x] Dynamic Resolution Scaling (DRS)
|
||||
- [ ] Draw camera frustums from the DatasetInputs
|
||||
|
||||
## Acknowledgements and references
|
||||
|
||||
We thank [Robin Deits](https://github.com/rdeits) and other contributors to the following repos, which we've started with and modified and extended for our use.
|
||||
|
||||
- [meshcat-python](https://github.com/rdeits/meshcat-python)
|
||||
- [meshcat](https://github.com/rdeits/meshcat)
|
||||
|
||||
Here are other resources that we've used and/or have found helpful while building the viewer code.
|
||||
|
||||
- [ThreeJS](https://threejs.org/)
|
||||
- [ReactJS](https://reactjs.org/)
|
||||
- [WebRTC](https://webrtc.org/) - WebRTC is a framework for real-time communication that allows two peers to send video, audio, or general data to and from each other with low latency. We've adopted WebRTC to stream rendered images to our viewer.
|
||||
@@ -12,6 +12,8 @@ jupyterlab==3.3.4
|
||||
kaleido
|
||||
matplotlib
|
||||
mediapy
|
||||
msgpack
|
||||
msgpack_numpy
|
||||
opencv-python
|
||||
plotly==5.7.0
|
||||
protobuf==3.20.0
|
||||
|
||||
@@ -52,7 +52,7 @@
|
||||
"from pyrad.utils.config import Config, setup_config\n",
|
||||
"\n",
|
||||
"# the new import\n",
|
||||
"from pyrad.viewer.backend import vis_utils\n",
|
||||
"from pyrad.viewer.server import vis_utils\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"from hydra import compose, initialize\n",
|
||||
@@ -61,8 +61,12 @@
|
||||
"from tqdm import tqdm\n",
|
||||
"import random\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"import umsgpack\n",
|
||||
"from pyrad.viewer.backend.utils import get_chunks, get_intrinsics_matrix_and_camera_to_world_h"
|
||||
"from pyrad.viewer.server.utils import get_chunks, get_intrinsics_matrix_and_camera_to_world_h\n",
|
||||
"\n",
|
||||
"from pyrad.viewer.server.visualizer import Viewer"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -78,7 +82,7 @@
|
||||
" overrides=[\n",
|
||||
" \"trainer.resume_train.load_dir=/shared/ethanweber/pyrad/outputs/blender_lego/instant_ngp/2022-07-05_200513/checkpoints\",\n",
|
||||
" \"trainer.resume_train.load_step=5000\",\n",
|
||||
" \"viewer.render_image_height=50\",\n",
|
||||
" # \"viewer.render_image_height=50\"\n",
|
||||
" ],\n",
|
||||
" )\n",
|
||||
" config = setup_config(config)"
|
||||
@@ -87,21 +91,11 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "5d231e36-dd38-489b-a3a3-860cdf49e551",
|
||||
"id": "26d6da3c-e28a-4c55-9592-d90dccb86051",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"2022-07-05 23:33:06.262342: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"trainer = Trainer(config, local_rank=0, world_size=1)\n",
|
||||
"trainer.setup()\n",
|
||||
"trainer.draw_scene_in_viewer()"
|
||||
"vis = Viewer(zmq_url=\"tcp://127.0.0.1:6000\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -127,7 +121,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"execution_count": 9,
|
||||
"id": "fd04703a-aa50-45e4-95fd-4d881ff18d61",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -136,7 +130,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"estimated_time: 3.3333333333333335\n",
|
||||
"time_elapsed: 14.046947002410889\n"
|
||||
"time_elapsed: 3.510408878326416\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -159,10 +153,10 @@
|
||||
"\n",
|
||||
"start = time.time()\n",
|
||||
"for camera in cameras:\n",
|
||||
" vis_utils.set_camera(trainer.vis, camera)\n",
|
||||
" vis_utils.set_camera(vis, camera)\n",
|
||||
"\n",
|
||||
" # render an image\n",
|
||||
" outputs = trainer.render_image_in_viewer()\n",
|
||||
" # outputs = trainer.render_image_in_viewer()\n",
|
||||
"\n",
|
||||
" time.sleep(1 / fps)\n",
|
||||
" # break\n",
|
||||
@@ -170,28 +164,21 @@
|
||||
"print(\"time_elapsed:\", time_elapsed)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"id": "fbb89f42-bf37-4786-a865-64c4ef018005",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"outputs = trainer.render_image_in_viewer()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "d91dfa73-ba0c-4412-a908-74ea51c37c58",
|
||||
"id": "0d937035-563c-412e-95ff-750ae54a42be",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "4a749acc7d255f078aee52e0584cc77b3cb5aaed1b3a7407ec4262c1bf6cb526"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.13 ('mattport')",
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
@@ -206,11 +193,6 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.13"
|
||||
},
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "f0938bb18d5c7de3ca7c6a73cb6be5f9b7603b44d115fc3e0726e01396d4d2be"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -32,7 +32,7 @@ from pyrad.utils import profiler, writer
|
||||
from pyrad.utils.config import Config
|
||||
from pyrad.utils.decorators import check_main_thread
|
||||
from pyrad.utils.writer import EventName, TimeWriter
|
||||
from pyrad.viewer.backend import vis_utils
|
||||
from pyrad.viewer.server import vis_utils
|
||||
|
||||
logging.getLogger("PIL").setLevel(logging.WARNING)
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ The Graph module contains all trainable parameters.
|
||||
from abc import abstractmethod
|
||||
from collections import defaultdict
|
||||
import time
|
||||
import traceback
|
||||
from typing import Any, Dict, List, Union
|
||||
|
||||
import torch
|
||||
@@ -33,7 +34,7 @@ from pyrad.graphs.modules.ray_generator import RayGenerator
|
||||
from pyrad.utils import profiler
|
||||
from pyrad.utils.config import GraphConfig
|
||||
from pyrad.utils.misc import get_masked_dict, instantiate_from_dict_config, is_not_none
|
||||
from pyrad.viewer.backend.vis_utils import CameraChangeException
|
||||
from pyrad.viewer.server.vis_utils import CameraChangeException
|
||||
|
||||
|
||||
@profiler.time_function
|
||||
@@ -197,6 +198,7 @@ class Graph(AbstractGraph):
|
||||
self.vis_outputs = outputs
|
||||
return outputs
|
||||
except CameraChangeException:
|
||||
print(traceback.format_exc())
|
||||
return None
|
||||
|
||||
def get_outputs_for_camera(self, camera: Camera):
|
||||
|
||||
@@ -104,10 +104,10 @@ class NGPGraph(Graph):
|
||||
# Hacky treatment of bins as points to allow us to merge uniform and pdf.
|
||||
ts_uniform = (ray_samples_uniform.bin_starts + ray_samples_uniform.bin_ends) / 2.0
|
||||
ts_pdf = (ray_samples_pdf.bin_starts + ray_samples_pdf.bin_ends) / 2.0
|
||||
ts, indices = torch.sort(torch.cat([ts_uniform, ts_pdf], -1), -1)
|
||||
ts, indices = torch.sort(torch.cat([ts_uniform, ts_pdf], -2), -2)
|
||||
bin_starts = ts
|
||||
bin_ends = torch.cat([ts[..., 1:], ts[..., -1:]], dim=-1)
|
||||
ray_samples = ray_bundle.get_ray_samples(bin_starts=bin_starts, bin_ends=bin_ends)
|
||||
bin_ends = torch.cat([ts[..., 1:, :], ts[..., -1:, :]], dim=-2)
|
||||
ray_samples = ray_bundle.get_ray_samples(bin_starts=bin_starts[..., 0], bin_ends=bin_ends[..., 0])
|
||||
|
||||
field_outputs = {}
|
||||
for fo_name, _ in field_outputs_pdf.items():
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
# Viewer
|
||||
|
||||
#### Backend (backend/)
|
||||
|
||||
The backend faciliates communication between Python code and the ReactJS client application.
|
||||
|
||||
```
|
||||
cd backend
|
||||
python run_zmq_server.py
|
||||
|
||||
It should print out something of the form:
|
||||
"ZMQWebSocketBridge using zmq_url=tcp://127.0.0.1:6000 and websocket_port=8051"
|
||||
```
|
||||
|
||||
#### Client (app/)
|
||||
|
||||
The ReactJS client code is in the `app` folder. First, install the most recent version of node.
|
||||
|
||||
```
|
||||
cd app
|
||||
|
||||
# install yarn and package.json dependencies
|
||||
npm install --global yarn
|
||||
yarn install
|
||||
|
||||
# start the application
|
||||
yarn start
|
||||
|
||||
# forward port and open in your local browser
|
||||
# the URL takes the following form
|
||||
# http://localhost:<forwarded_react_port>/localhost:<forwarded_backend_tcp_port>
|
||||
http://localhost:4000/localhost:8051
|
||||
```
|
||||
|
||||
Possibly edit the port in `app/.env.development` file with the following:
|
||||
|
||||
```
|
||||
BROWSER=none
|
||||
FAST_REFRESH=false
|
||||
HOST=localhost
|
||||
PORT=4000
|
||||
```
|
||||
|
||||
#### Establishing the connection from Python
|
||||
|
||||
See `notebooks/visualize_viewer.ipynb` for a quick overview of how to interact with the viewer.
|
||||
|
||||
<hr>
|
||||
|
||||
#### How to use with VNC
|
||||
|
||||
> This section is more experimental and shouldn't be used right now. Eventually it would be nice to support an Electron app for local use instead of using a web browser (e.g., Chrome).
|
||||
|
||||
```
|
||||
https://tigervnc.org/doc/Xvnc.html
|
||||
|
||||
# start vnc server
|
||||
/opt/TurboVNC/bin/Xvnc :5017 -rfbport 5017 -SecurityTypes none -localsts
|
||||
|
||||
# run the application. TODO: get electron to work!
|
||||
alias wgui='vglrun -d :0.0'
|
||||
export DISPLAY=:5017
|
||||
wgui chromium-browser http://localhost:3000/localhost:8051
|
||||
```
|
||||
|
||||
The server communication code is in the `comms` folder.
|
||||
Generated
+28176
File diff suppressed because it is too large.
Load diff
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"name": "viewer",
|
||||
"homepage": "https://ethanweber.me/pyrad-private/",
|
||||
"version": "0.0.0",
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
@@ -49,4 +50,4 @@
|
||||
"typescript": "^4.7.3",
|
||||
"wait-on": "^6.0.1"
|
||||
}
|
||||
}
|
||||
}
|
||||
+342
-362
@@ -8,387 +8,367 @@ import { SceneNode } from "./SceneNode";
|
||||
import { ExtensibleObjectLoader } from "./ExtensibleObjectLoader";
|
||||
|
||||
function websocket_endpoint_from_url(url) {
|
||||
let endpoint = url.split("/").pop();
|
||||
if (endpoint == "") {
|
||||
let message =
|
||||
"Please set the websocket endpoint. E.g., a correct URL may be: http://localhost:4000/localhost:8051";
|
||||
window.alert(message);
|
||||
return null;
|
||||
}
|
||||
return endpoint;
|
||||
let endpoint = url.split("?").pop();
|
||||
if (endpoint == "") {
|
||||
let message =
|
||||
"Please set the websocket endpoint. E.g., a correct URL may be: http://localhost:4000?localhost:8051";
|
||||
window.alert(message);
|
||||
return null;
|
||||
}
|
||||
return endpoint;
|
||||
}
|
||||
|
||||
function findCameraObjectUnderObject3D(object) {
|
||||
// seach the tree for the camera
|
||||
if (object instanceof THREE.Camera) {
|
||||
return object;
|
||||
}
|
||||
for (let name of Object.keys(object.children)) {
|
||||
return findCameraObjectUnderObject3D(object.children[name]);
|
||||
}
|
||||
return null;
|
||||
// seach the tree for the camera
|
||||
if (object instanceof THREE.Camera) {
|
||||
return object;
|
||||
}
|
||||
for (let name of Object.keys(object.children)) {
|
||||
return findCameraObjectUnderObject3D(object.children[name]);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
var msgpack = require("msgpack-lite");
|
||||
|
||||
export class Viewer extends Component {
|
||||
constructor(props) {
|
||||
super(props);
|
||||
this.state = {
|
||||
scene: null,
|
||||
controls: null,
|
||||
renderer: null,
|
||||
gui: null,
|
||||
scene_tree: null,
|
||||
viewport_width: null,
|
||||
viewport_height: null,
|
||||
images: [],
|
||||
image_index: 0,
|
||||
};
|
||||
this.update = this.update.bind(this);
|
||||
this.set_object = this.set_object.bind(this);
|
||||
this.handle_command = this.handle_command.bind(this);
|
||||
this.threejs_ref = React.createRef();
|
||||
}
|
||||
|
||||
handleResize() {
|
||||
this.state.viewport_width = this.getViewportWidth();
|
||||
this.state.viewport_height = this.getViewportHeight();
|
||||
this.state.camera_main.aspect =
|
||||
this.state.viewport_width / this.state.viewport_height;
|
||||
this.state.camera_main.updateProjectionMatrix();
|
||||
this.state.renderer_main.setSize(
|
||||
this.state.viewport_width,
|
||||
this.state.viewport_height
|
||||
);
|
||||
}
|
||||
|
||||
update() {
|
||||
requestAnimationFrame(this.update);
|
||||
this.handleResize();
|
||||
let camera_main = findCameraObjectUnderObject3D(
|
||||
this.state.scene_tree.find(["Cameras", "Main Camera"]).object
|
||||
);
|
||||
camera_main.updateProjectionMatrix();
|
||||
if (this.state.controls_main != null) {
|
||||
this.state.controls_main.update();
|
||||
}
|
||||
this.state.renderer_main.render(this.state.scene, camera_main);
|
||||
this.send_camera_over_websocket();
|
||||
}
|
||||
|
||||
send_camera_over_websocket() {
|
||||
/* update the camera information in the python server
|
||||
if the websocket is connected */
|
||||
console.log("send_camera_over_websocket");
|
||||
if (this.state.websocket.readyState === WebSocket.OPEN) {
|
||||
// update the camera information in the python server
|
||||
let camera_main = findCameraObjectUnderObject3D(
|
||||
this.state.scene_tree.find(["Cameras", "Main Camera"]).object
|
||||
);
|
||||
console.log(camera_main.toJSON());
|
||||
let cmd = "set_object";
|
||||
let path = "Cameras/Main Camera";
|
||||
let data = {
|
||||
type: cmd,
|
||||
path: path,
|
||||
object: camera_main.toJSON(),
|
||||
};
|
||||
let message = msgpack.encode(data);
|
||||
this.state.websocket.send(message);
|
||||
}
|
||||
}
|
||||
|
||||
save_image(image) {
|
||||
let url = "https://friends.ethanweber.me/save_image";
|
||||
let data = {
|
||||
index: this.state.image_index,
|
||||
image: image,
|
||||
};
|
||||
fetch(url, {
|
||||
method: "POST", // The method
|
||||
mode: "no-cors", // It can be no-cors, cors, same-origin
|
||||
headers: {
|
||||
"Content-Type": "application/json", // Your headers
|
||||
},
|
||||
body: JSON.stringify(data),
|
||||
}).then((returned) => {
|
||||
console.log(returned);
|
||||
});
|
||||
this.state.image_index += 1;
|
||||
}
|
||||
|
||||
set_object(path, object) {
|
||||
this.state.scene_tree.find(path.concat(["<object>"])).set_object(object);
|
||||
// add controls if object is a camera
|
||||
if (object instanceof THREE.Camera) {
|
||||
this.state.controls_main = new TrackballControls(
|
||||
object,
|
||||
this.state.renderer_main.domElement
|
||||
);
|
||||
this.state.controls_main.rotateSpeed = 2.0;
|
||||
this.state.controls_main.zoomSpeed = 0.3;
|
||||
this.state.controls_main.panSpeed = 0.2;
|
||||
this.state.controls_main.staticMoving = false; // false is default
|
||||
this.state.controls_main.target.set(0, 0, 0); // focus point of the controls
|
||||
this.state.controls_main.autoRotate = false;
|
||||
this.state.controls_main.dynamicDampingFactor = 1.0;
|
||||
this.state.controls_main.update();
|
||||
// this.send_camera_over_websocket();
|
||||
// this.state.controls_main.addEventListener("change", () => {
|
||||
// this.send_camera_over_websocket();
|
||||
// });
|
||||
}
|
||||
}
|
||||
|
||||
set_object_from_json(path, object_json) {
|
||||
let loader = new ExtensibleObjectLoader();
|
||||
loader.parse(object_json, (obj) => {
|
||||
if (
|
||||
obj.geometry !== undefined &&
|
||||
obj.geometry.type === "BufferGeometry"
|
||||
) {
|
||||
if (
|
||||
obj.geometry.attributes.normal === undefined ||
|
||||
obj.geometry.attributes.normal.count === 0
|
||||
) {
|
||||
obj.geometry.computeVertexNormals();
|
||||
}
|
||||
}
|
||||
obj.castShadow = true;
|
||||
obj.receiveShadow = true;
|
||||
this.set_object(path, obj);
|
||||
});
|
||||
}
|
||||
|
||||
set_transform(path, matrix) {
|
||||
this.state.scene_tree.find(path).set_transform(matrix);
|
||||
}
|
||||
|
||||
delete_path(path) {
|
||||
if (path.length === 0) {
|
||||
console.error("Deleting the entire scene is not implemented.");
|
||||
} else {
|
||||
this.state.scene_tree.delete(path);
|
||||
}
|
||||
}
|
||||
|
||||
set_property(path, property, value) {
|
||||
this.state.scene_tree.find(path).set_property(property, value);
|
||||
// TODO(ethan): handle this issue
|
||||
if (path[0] === "Background") {
|
||||
// The background is not an Object3d, so needs a little help.
|
||||
this.state.scene_tree.find(path).on_update();
|
||||
}
|
||||
}
|
||||
|
||||
handle_command(cmd) {
|
||||
// convert binary serialization format back to JSON
|
||||
cmd = msgpack.decode(new Uint8Array(cmd));
|
||||
console.log(cmd);
|
||||
|
||||
// three js scene commands
|
||||
if (cmd.type === "set_object") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_object_from_json(path, cmd.object);
|
||||
} else if (cmd.type === "set_transform") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_transform(path, cmd.matrix);
|
||||
} else if (cmd.type === "delete") {
|
||||
let path = split_path(cmd.path);
|
||||
this.delete_path(path);
|
||||
} else if (cmd.type === "set_property") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_property(path, cmd.property, cmd.value);
|
||||
}
|
||||
// web rtc commands
|
||||
else if (cmd.type === "answer") {
|
||||
let answer = cmd.data;
|
||||
this.state.pc.setRemoteDescription(answer);
|
||||
}
|
||||
|
||||
// possibly update the camera information in the python server
|
||||
// this.send_camera_over_websocket();
|
||||
}
|
||||
|
||||
getViewportWidth() {
|
||||
return window.innerWidth - (window.innerWidth % 2);
|
||||
}
|
||||
|
||||
getViewportHeight() {
|
||||
return window.innerHeight;
|
||||
}
|
||||
|
||||
setupWebRTC() {
|
||||
console.log("setting up WebRTC");
|
||||
// The iceServers config comes from the following URL:
|
||||
// https://www.metered.ca/tools/openrelay/
|
||||
this.state.pc = new RTCPeerConnection({
|
||||
iceServers: [
|
||||
{
|
||||
urls: "stun:openrelay.metered.ca:80",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:80",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:443",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:443?transport=tcp",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
// connect video
|
||||
this.state.pc.addEventListener("track", function (evt) {
|
||||
if (evt.track.kind == "video") {
|
||||
console.log("setting event stream");
|
||||
document.getElementById("WebRTCVideo-video").srcObject = evt.streams[0];
|
||||
}
|
||||
});
|
||||
this.state.pc.addTransceiver("video", { direction: "recvonly" });
|
||||
|
||||
this.state.pc
|
||||
.createOffer()
|
||||
.then((offer) => {
|
||||
return this.state.pc.setLocalDescription(offer);
|
||||
})
|
||||
.then(() => {
|
||||
// wait for ICE gathering to complete
|
||||
return new Promise((resolve) => {
|
||||
if (this.state.pc.iceGatheringState === "complete") {
|
||||
resolve();
|
||||
} else {
|
||||
var checkState = () => {
|
||||
if (this.state.pc.iceGatheringState === "complete") {
|
||||
this.state.pc.removeEventListener(
|
||||
"icegatheringstatechange",
|
||||
checkState
|
||||
);
|
||||
resolve();
|
||||
}
|
||||
};
|
||||
this.state.pc.addEventListener(
|
||||
"icegatheringstatechange",
|
||||
checkState
|
||||
);
|
||||
}
|
||||
});
|
||||
})
|
||||
.then(() => {
|
||||
var offer = this.state.pc.localDescription;
|
||||
console.log("sending the offer");
|
||||
let cmd = "offer";
|
||||
let path = "";
|
||||
let data = {
|
||||
type: cmd,
|
||||
path: path,
|
||||
data: {
|
||||
sdp: offer.sdp,
|
||||
type: offer.type,
|
||||
},
|
||||
constructor(props) {
|
||||
super(props);
|
||||
this.state = {
|
||||
scene: null,
|
||||
controls: null,
|
||||
renderer: null,
|
||||
gui: null,
|
||||
scene_tree: null,
|
||||
viewport_width: null,
|
||||
viewport_height: null,
|
||||
};
|
||||
let message = msgpack.encode(data);
|
||||
this.state.websocket.send(message);
|
||||
});
|
||||
}
|
||||
this.update = this.update.bind(this);
|
||||
this.set_object = this.set_object.bind(this);
|
||||
this.handle_command = this.handle_command.bind(this);
|
||||
this.threejs_ref = React.createRef();
|
||||
}
|
||||
|
||||
componentDidMount() {
|
||||
// Open the websocket
|
||||
let endpoint = websocket_endpoint_from_url(window.location.href);
|
||||
this.state.websocket = new WebSocket("ws://" + endpoint + "/");
|
||||
this.state.websocket.onopen = () => {
|
||||
console.log("websocket connected");
|
||||
this.setupWebRTC();
|
||||
};
|
||||
this.state.websocket.binaryType = "arraybuffer";
|
||||
// 'command' for updates to Viewer
|
||||
this.state.websocket.onmessage = (cmd) => this.handle_command(cmd.data);
|
||||
this.state.websocket.onclose = function (evt) {
|
||||
console.log("onclose:", evt);
|
||||
};
|
||||
handleResize() {
|
||||
this.state.viewport_width = this.getViewportWidth();
|
||||
this.state.viewport_height = this.getViewportHeight();
|
||||
this.state.camera_main.aspect =
|
||||
this.state.viewport_width / this.state.viewport_height;
|
||||
this.state.camera_main.updateProjectionMatrix();
|
||||
this.state.renderer_main.setSize(
|
||||
this.state.viewport_width,
|
||||
this.state.viewport_height
|
||||
);
|
||||
}
|
||||
|
||||
// Get size of screen
|
||||
this.state.viewport_width = this.getViewportWidth();
|
||||
this.state.viewport_height = this.getViewportHeight();
|
||||
update() {
|
||||
requestAnimationFrame(this.update);
|
||||
this.handleResize();
|
||||
let camera_main = findCameraObjectUnderObject3D(
|
||||
this.state.scene_tree.find(["Cameras", "Main Camera"]).object
|
||||
);
|
||||
camera_main.updateProjectionMatrix();
|
||||
if (this.state.controls_main != null) {
|
||||
this.state.controls_main.update();
|
||||
}
|
||||
this.state.renderer_main.render(this.state.scene, camera_main);
|
||||
this.send_camera_over_websocket();
|
||||
}
|
||||
|
||||
// Scene
|
||||
this.state.scene = new THREE.Scene();
|
||||
send_camera_over_websocket() {
|
||||
/* update the camera information in the python server
|
||||
if the websocket is connected */
|
||||
console.log("send_camera_over_websocket");
|
||||
if (this.state.websocket.readyState === WebSocket.OPEN) {
|
||||
// update the camera information in the python server
|
||||
let camera_main = findCameraObjectUnderObject3D(
|
||||
this.state.scene_tree.find(["Cameras", "Main Camera"]).object
|
||||
);
|
||||
console.log(camera_main.toJSON());
|
||||
let cmd = "set_object";
|
||||
let path = "Cameras/Main Camera";
|
||||
let data = {
|
||||
type: cmd,
|
||||
path: path,
|
||||
object: camera_main.toJSON(),
|
||||
};
|
||||
let message = msgpack.encode(data);
|
||||
this.state.websocket.send(message);
|
||||
}
|
||||
}
|
||||
|
||||
// GUI
|
||||
this.state.gui = new GUI();
|
||||
let scene_folder = this.state.gui.addFolder("Scene");
|
||||
scene_folder.open();
|
||||
this.state.scene_tree = new SceneNode(this.state.scene, scene_folder);
|
||||
set_object(path, object) {
|
||||
this.state.scene_tree.find(path.concat(["<object>"])).set_object(object);
|
||||
// add controls if object is a camera
|
||||
if (object instanceof THREE.Camera) {
|
||||
this.state.controls_main = new TrackballControls(
|
||||
object,
|
||||
this.state.renderer_main.domElement
|
||||
);
|
||||
this.state.controls_main.rotateSpeed = 2.0;
|
||||
this.state.controls_main.zoomSpeed = 0.3;
|
||||
this.state.controls_main.panSpeed = 0.2;
|
||||
this.state.controls_main.staticMoving = false; // false is default
|
||||
this.state.controls_main.target.set(0, 0, 0); // focus point of the controls
|
||||
this.state.controls_main.autoRotate = false;
|
||||
this.state.controls_main.dynamicDampingFactor = 1.0;
|
||||
this.state.controls_main.update();
|
||||
// this.send_camera_over_websocket();
|
||||
// this.state.controls_main.addEventListener("change", () => {
|
||||
// this.send_camera_over_websocket();
|
||||
// });
|
||||
}
|
||||
}
|
||||
|
||||
// Renderer main
|
||||
this.state.renderer_main = new THREE.WebGLRenderer({
|
||||
antialias: true,
|
||||
alpha: true,
|
||||
});
|
||||
this.state.renderer_main.setPixelRatio(window.devicePixelRatio);
|
||||
this.state.renderer_main.setSize(
|
||||
this.state.viewport_width,
|
||||
this.state.viewport_height
|
||||
);
|
||||
this.state.renderer_main.domElement.style.border = "1px solid black";
|
||||
set_object_from_json(path, object_json) {
|
||||
let loader = new ExtensibleObjectLoader();
|
||||
loader.parse(object_json, (obj) => {
|
||||
if (
|
||||
obj.geometry !== undefined &&
|
||||
obj.geometry.type === "BufferGeometry"
|
||||
) {
|
||||
if (
|
||||
obj.geometry.attributes.normal === undefined ||
|
||||
obj.geometry.attributes.normal.count === 0
|
||||
) {
|
||||
obj.geometry.computeVertexNormals();
|
||||
}
|
||||
}
|
||||
obj.castShadow = true;
|
||||
obj.receiveShadow = true;
|
||||
this.set_object(path, obj);
|
||||
});
|
||||
}
|
||||
|
||||
let domElement = document.getElementById("canvas-container-main");
|
||||
domElement.append(this.state.renderer_main.domElement);
|
||||
set_transform(path, matrix) {
|
||||
this.state.scene_tree.find(path).set_transform(matrix);
|
||||
}
|
||||
|
||||
// Camera main
|
||||
this.state.camera_main = new THREE.PerspectiveCamera(
|
||||
120,
|
||||
this.state.viewport_width / this.state.viewport_height,
|
||||
0.01,
|
||||
100
|
||||
);
|
||||
this.state.camera_main.position.x = 5;
|
||||
this.state.camera_main.position.y = -5;
|
||||
this.state.camera_main.position.z = -5;
|
||||
this.state.camera_main.up = new THREE.Vector3(0, -1, 0);
|
||||
this.set_object(["Cameras", "Main Camera"], this.state.camera_main);
|
||||
delete_path(path) {
|
||||
if (path.length === 0) {
|
||||
console.error("Deleting the entire scene is not implemented.");
|
||||
} else {
|
||||
this.state.scene_tree.delete(path);
|
||||
}
|
||||
}
|
||||
|
||||
// Axes display
|
||||
let axes = new THREE.AxesHelper(5);
|
||||
this.set_object(["Axes"], axes);
|
||||
set_property(path, property, value) {
|
||||
this.state.scene_tree.find(path).set_property(property, value);
|
||||
// TODO(ethan): handle this issue
|
||||
if (path[0] === "Background") {
|
||||
// The background is not an Object3d, so needs a little help.
|
||||
this.state.scene_tree.find(path).on_update();
|
||||
}
|
||||
}
|
||||
|
||||
// Grid
|
||||
let grid = new THREE.GridHelper(20, 20);
|
||||
this.set_object(["Grid"], grid);
|
||||
handle_command(cmd) {
|
||||
// convert binary serialization format back to JSON
|
||||
cmd = msgpack.decode(new Uint8Array(cmd));
|
||||
console.log(cmd);
|
||||
|
||||
// Lights
|
||||
let color = 0xffffff;
|
||||
let intensity = 1;
|
||||
let light = new THREE.AmbientLight(color, intensity);
|
||||
this.set_object(["Light"], light);
|
||||
// three js scene commands
|
||||
if (cmd.type === "set_object") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_object_from_json(path, cmd.object);
|
||||
} else if (cmd.type === "set_transform") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_transform(path, cmd.matrix);
|
||||
} else if (cmd.type === "delete") {
|
||||
let path = split_path(cmd.path);
|
||||
this.delete_path(path);
|
||||
} else if (cmd.type === "set_property") {
|
||||
let path = split_path(cmd.path);
|
||||
this.set_property(path, cmd.property, cmd.value);
|
||||
}
|
||||
// web rtc commands
|
||||
else if (cmd.type === "answer") {
|
||||
let answer = cmd.data;
|
||||
this.state.pc.setRemoteDescription(answer);
|
||||
}
|
||||
|
||||
// Stats
|
||||
var stats = new Stats();
|
||||
let domElement_stats = document.getElementById("canvas-container-main");
|
||||
domElement_stats.append(stats.dom);
|
||||
// possibly update the camera information in the python server
|
||||
// this.send_camera_over_websocket();
|
||||
}
|
||||
|
||||
this.update();
|
||||
}
|
||||
getViewportWidth() {
|
||||
return window.innerWidth - (window.innerWidth % 2);
|
||||
}
|
||||
|
||||
render() {
|
||||
return (
|
||||
<div>
|
||||
<div className="WebRTCVideo">
|
||||
<video id="WebRTCVideo-video" autoPlay playsInline muted>
|
||||
{" "}
|
||||
</video>{" "}
|
||||
</div>{" "}
|
||||
<div id="canvas-container-main"> </div>{" "}
|
||||
<div id="stats-container"> </div>{" "}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
}
|
||||
getViewportHeight() {
|
||||
return window.innerHeight;
|
||||
}
|
||||
|
||||
setupWebRTC() {
|
||||
console.log("setting up WebRTC");
|
||||
// The iceServers config comes from the following URL:
|
||||
// https://www.metered.ca/tools/openrelay/
|
||||
this.state.pc = new RTCPeerConnection({
|
||||
iceServers: [{
|
||||
urls: "stun:openrelay.metered.ca:80",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:80",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:443",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
{
|
||||
urls: "turn:openrelay.metered.ca:443?transport=tcp",
|
||||
username: "openrelayproject",
|
||||
credential: "openrelayproject",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
// connect video
|
||||
this.state.pc.addEventListener("track", function(evt) {
|
||||
if (evt.track.kind == "video") {
|
||||
console.log("setting event stream");
|
||||
document.getElementById("WebRTCVideo-video").srcObject = evt.streams[0];
|
||||
}
|
||||
});
|
||||
this.state.pc.addTransceiver("video", { direction: "recvonly" });
|
||||
|
||||
this.state.pc
|
||||
.createOffer()
|
||||
.then((offer) => {
|
||||
return this.state.pc.setLocalDescription(offer);
|
||||
})
|
||||
.then(() => {
|
||||
// wait for ICE gathering to complete
|
||||
return new Promise((resolve) => {
|
||||
if (this.state.pc.iceGatheringState === "complete") {
|
||||
resolve();
|
||||
} else {
|
||||
var checkState = () => {
|
||||
if (this.state.pc.iceGatheringState === "complete") {
|
||||
this.state.pc.removeEventListener(
|
||||
"icegatheringstatechange",
|
||||
checkState
|
||||
);
|
||||
resolve();
|
||||
}
|
||||
};
|
||||
this.state.pc.addEventListener(
|
||||
"icegatheringstatechange",
|
||||
checkState
|
||||
);
|
||||
}
|
||||
});
|
||||
})
|
||||
.then(() => {
|
||||
var offer = this.state.pc.localDescription;
|
||||
console.log("sending the offer");
|
||||
let cmd = "offer";
|
||||
let path = "";
|
||||
let data = {
|
||||
type: cmd,
|
||||
path: path,
|
||||
data: {
|
||||
sdp: offer.sdp,
|
||||
type: offer.type,
|
||||
},
|
||||
};
|
||||
let message = msgpack.encode(data);
|
||||
this.state.websocket.send(message);
|
||||
});
|
||||
}
|
||||
|
||||
componentDidMount() {
|
||||
// Open the websocket
|
||||
let endpoint = websocket_endpoint_from_url(window.location.href);
|
||||
this.state.websocket = new WebSocket("ws://" + endpoint + "/");
|
||||
this.state.websocket.onopen = () => {
|
||||
console.log("websocket connected");
|
||||
this.setupWebRTC();
|
||||
};
|
||||
this.state.websocket.binaryType = "arraybuffer";
|
||||
// 'command' for updates to Viewer
|
||||
this.state.websocket.onmessage = (cmd) => this.handle_command(cmd.data);
|
||||
this.state.websocket.onclose = function(evt) {
|
||||
console.log("onclose:", evt);
|
||||
};
|
||||
|
||||
// Get size of screen
|
||||
this.state.viewport_width = this.getViewportWidth();
|
||||
this.state.viewport_height = this.getViewportHeight();
|
||||
|
||||
// Scene
|
||||
this.state.scene = new THREE.Scene();
|
||||
|
||||
// GUI
|
||||
this.state.gui = new GUI();
|
||||
let scene_folder = this.state.gui.addFolder("Scene");
|
||||
scene_folder.open();
|
||||
this.state.scene_tree = new SceneNode(this.state.scene, scene_folder);
|
||||
|
||||
// Renderer main
|
||||
this.state.renderer_main = new THREE.WebGLRenderer({
|
||||
antialias: true,
|
||||
alpha: true,
|
||||
});
|
||||
this.state.renderer_main.setPixelRatio(window.devicePixelRatio);
|
||||
this.state.renderer_main.setSize(
|
||||
this.state.viewport_width,
|
||||
this.state.viewport_height
|
||||
);
|
||||
this.state.renderer_main.domElement.style.border = "1px solid black";
|
||||
|
||||
let domElement = document.getElementById("canvas-container-main");
|
||||
domElement.append(this.state.renderer_main.domElement);
|
||||
|
||||
// Camera main
|
||||
this.state.camera_main = new THREE.PerspectiveCamera(
|
||||
120,
|
||||
this.state.viewport_width / this.state.viewport_height,
|
||||
0.01,
|
||||
100
|
||||
);
|
||||
this.state.camera_main.position.x = 5;
|
||||
this.state.camera_main.position.y = -5;
|
||||
this.state.camera_main.position.z = 5;
|
||||
this.state.camera_main.up = new THREE.Vector3(0, 0, 1);
|
||||
this.set_object(["Cameras", "Main Camera"], this.state.camera_main);
|
||||
|
||||
// Axes display
|
||||
let axes = new THREE.AxesHelper(5);
|
||||
this.set_object(["Axes"], axes);
|
||||
|
||||
// Grid
|
||||
let grid = new THREE.GridHelper(20, 20);
|
||||
this.set_object(["Grid"], grid);
|
||||
|
||||
// Lights
|
||||
let color = 0xffffff;
|
||||
let intensity = 1;
|
||||
let light = new THREE.AmbientLight(color, intensity);
|
||||
this.set_object(["Light"], light);
|
||||
|
||||
// Stats
|
||||
var stats = new Stats();
|
||||
let domElement_stats = document.getElementById("canvas-container-main");
|
||||
domElement_stats.append(stats.dom);
|
||||
|
||||
this.update();
|
||||
}
|
||||
|
||||
render() {
|
||||
return ( <
|
||||
div >
|
||||
<
|
||||
div className = "WebRTCVideo" >
|
||||
<
|
||||
video id = "WebRTCVideo-video"
|
||||
autoPlay playsInline muted > { " " } <
|
||||
/video>{" "} <
|
||||
/div>{" "} <
|
||||
div id = "canvas-container-main" > < /div>{" "} <
|
||||
div id = "stats-container" > < /div>{" "} <
|
||||
/div>
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -18,7 +18,7 @@ from . import commands
|
||||
from . import geometry
|
||||
from . import visualizer
|
||||
from . import transformations
|
||||
from .visualizer import ViewerWindow, Visualizer
|
||||
from .visualizer import ViewerWindow, Viewer
|
||||
|
||||
|
||||
def viewer_assets_path():
|
||||
File renamed without changes.
@@ -68,26 +68,6 @@ class GetObject:
|
||||
return {"type": "get_object", "path": self.path.lower()}
|
||||
|
||||
|
||||
class SetImage:
|
||||
"""
|
||||
Sets the image for showing with web rtc.
|
||||
"""
|
||||
|
||||
__slots__ = ["image", "path"]
|
||||
|
||||
def __init__(self, image, path):
|
||||
self.image = image
|
||||
self.path = path
|
||||
|
||||
def lower(self):
|
||||
return {
|
||||
"type": "set_image",
|
||||
"path": self.path.lower(),
|
||||
"image": list(self.image.flatten()),
|
||||
"shape": list(self.image.shape),
|
||||
}
|
||||
|
||||
|
||||
class SetTransform:
|
||||
__slots__ = ["matrix", "path"]
|
||||
|
||||
File renamed without changes.
File renamed without changes.
@@ -14,60 +14,25 @@
|
||||
|
||||
from __future__ import absolute_import, division, print_function
|
||||
|
||||
import base64
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
|
||||
if sys.version_info >= (3, 0):
|
||||
ADDRESS_IN_USE_ERROR = OSError
|
||||
else:
|
||||
import socket
|
||||
|
||||
ADDRESS_IN_USE_ERROR = socket.error
|
||||
|
||||
import tornado.gen
|
||||
import tornado.ioloop
|
||||
import tornado.web
|
||||
import tornado.websocket
|
||||
import msgpack
|
||||
import msgpack_numpy
|
||||
import umsgpack
|
||||
import zmq
|
||||
import zmq.eventloop.ioloop
|
||||
from aiortc import RTCPeerConnection, RTCSessionDescription, VideoStreamTrack
|
||||
from aiortc import RTCPeerConnection, RTCSessionDescription
|
||||
from aiortc.rtcrtpsender import RTCRtpSender
|
||||
from zmq.eventloop.zmqstream import ZMQStream
|
||||
from pyrad.viewer.server.socket import SerializingContext
|
||||
|
||||
from pyrad.viewer.backend.tree import SceneTree, find_node, walk
|
||||
from pyrad.viewer.backend.video_stream import FlagVideoStreamTrack, SingleFrameStreamTrack
|
||||
|
||||
|
||||
def capture(pattern, s):
|
||||
match = re.match(pattern, s)
|
||||
if not match:
|
||||
raise ValueError("Could not match {:s} with pattern {:s}".format(s, pattern))
|
||||
else:
|
||||
return match.groups()[0]
|
||||
|
||||
|
||||
def match_zmq_url(line):
|
||||
return capture(r"^zmq_url=(.*)$", line)
|
||||
|
||||
|
||||
def _zmq_install_ioloop():
|
||||
# For pyzmq<17, install ioloop instead of a tornado ioloop
|
||||
# http://zeromq.github.com/pyzmq/eventloop.html
|
||||
try:
|
||||
pyzmq_major = int(zmq.__version__.split(".")[0])
|
||||
except ValueError:
|
||||
# Development version?
|
||||
return
|
||||
if pyzmq_major < 17:
|
||||
zmq.eventloop.ioloop.install()
|
||||
|
||||
|
||||
_zmq_install_ioloop()
|
||||
|
||||
from pyrad.viewer.server.tree import SceneTree, find_node, walk
|
||||
from pyrad.viewer.server.video_stream import SingleFrameStreamTrack
|
||||
|
||||
MAX_ATTEMPTS = 1000
|
||||
DEFAULT_ZMQ_METHOD = "tcp"
|
||||
@@ -82,7 +47,7 @@ def find_available_port(func, default_port, max_attempts=MAX_ATTEMPTS, **kwargs)
|
||||
port = default_port + i
|
||||
try:
|
||||
return func(port, **kwargs), port
|
||||
except (ADDRESS_IN_USE_ERROR, zmq.error.ZMQError):
|
||||
except (OSError, zmq.error.ZMQError):
|
||||
print("Port: {:d} in use, trying another...".format(port), file=sys.stderr)
|
||||
except Exception as e:
|
||||
print(type(e))
|
||||
@@ -105,12 +70,14 @@ def force_codec(pc, sender, forced_codec):
|
||||
|
||||
|
||||
class WebSocketHandler(tornado.websocket.WebSocketHandler):
|
||||
"""Tornado websocket handler for receiving and sending commands from/to the viewer."""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self.bridge = kwargs.pop("bridge")
|
||||
super(WebSocketHandler, self).__init__(*args, **kwargs)
|
||||
|
||||
# this disables CORS
|
||||
def check_origin(self, origin):
|
||||
"""This disables CORS."""
|
||||
return True
|
||||
|
||||
def open(self):
|
||||
@@ -119,7 +86,6 @@ class WebSocketHandler(tornado.websocket.WebSocketHandler):
|
||||
self.bridge.send_scene(self)
|
||||
|
||||
async def on_message(self, message):
|
||||
|
||||
data = message
|
||||
m = umsgpack.unpackb(message)
|
||||
type_ = m["type"]
|
||||
@@ -131,20 +97,15 @@ class WebSocketHandler(tornado.websocket.WebSocketHandler):
|
||||
find_node(self.bridge.tree, path).object = data
|
||||
find_node(self.bridge.tree, path).properties = []
|
||||
elif type_ == "offer":
|
||||
print("making an offer")
|
||||
print("sending an answer")
|
||||
# print(m)
|
||||
|
||||
offer = RTCSessionDescription(m["data"]["sdp"], m["data"]["type"])
|
||||
|
||||
pc = RTCPeerConnection()
|
||||
self.bridge.pcs.add(pc) # TODO(ethan): handle this better, since this set will get large
|
||||
self.bridge.pcs.add(pc)
|
||||
|
||||
# video = FlagVideoStreamTrack()
|
||||
video = SingleFrameStreamTrack()
|
||||
self.bridge.video_tracks.add(video)
|
||||
video_sender = pc.addTrack(video)
|
||||
# force_codec(this.bridge.pc, video_sender, video_codec)
|
||||
# force_codec(pc, video_sender, video_codec)
|
||||
|
||||
await pc.setRemoteDescription(offer)
|
||||
answer = await pc.createAnswer()
|
||||
@@ -165,6 +126,7 @@ class WebSocketHandler(tornado.websocket.WebSocketHandler):
|
||||
|
||||
class ZMQWebSocketBridge(object):
|
||||
context = zmq.Context()
|
||||
# context = SerializingContext()
|
||||
|
||||
def __init__(self, zmq_url=None, host="127.0.0.1", websocket_port=None):
|
||||
self.host = host
|
||||
@@ -219,6 +181,8 @@ class ZMQWebSocketBridge(object):
|
||||
find_node(self.tree, path).properties = []
|
||||
elif cmd == "get_object":
|
||||
data = find_node(self.tree, path).object
|
||||
if isinstance(data, type(None)):
|
||||
data = umsgpack.packb("error: object not found")
|
||||
self.zmq_socket.send(data)
|
||||
return
|
||||
elif cmd == "set_property":
|
||||
@@ -233,9 +197,10 @@ class ZMQWebSocketBridge(object):
|
||||
self.tree = SceneTree()
|
||||
elif cmd in WEBRTC_COMMANDS:
|
||||
if cmd == "set_image":
|
||||
image = msgpack.unpackb(
|
||||
data, object_hook=msgpack_numpy.decode, use_list=False, max_bin_len=50000000, raw=False
|
||||
)
|
||||
for video_track in self.video_tracks:
|
||||
unpacked_data = umsgpack.unpackb(data)
|
||||
image = np.array(unpacked_data["image"], dtype="uint8").reshape(unpacked_data["shape"])
|
||||
video_track.put_frame(image)
|
||||
else:
|
||||
self.zmq_socket.send(b"error: unknown command")
|
||||
@@ -270,20 +235,6 @@ class ZMQWebSocketBridge(object):
|
||||
self.ioloop.start()
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description="Listen for ZeroMQ commands")
|
||||
parser.add_argument("--zmq-url", "-z", type=str, nargs="?", default=None)
|
||||
parser.add_argument("--websocket-port", "-wp", type=str, nargs="?", default=None)
|
||||
args = parser.parse_args()
|
||||
bridge = ZMQWebSocketBridge(zmq_url=args.zmq_url, websocket_port=args.websocket_port)
|
||||
print(bridge)
|
||||
try:
|
||||
bridge.run()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
def start_server_as_subprocess(zmq_url=None):
|
||||
"""Starts the ZMQWebSocketBridge server as a subprocess."""
|
||||
raise NotImplementedError()
|
||||
@@ -0,0 +1,127 @@
|
||||
# https://gist.github.com/jeffbass/ebf877e964c9a0b84272
|
||||
|
||||
# zmqimage.py -- classes to send, receive and display cv2 images via zmq
|
||||
# based on serialization in pyzmq docs and pyzmq/examples/serialization
|
||||
|
||||
"""
|
||||
PURPOSE:
|
||||
These classes allow a headless (no display) computer running OpenCV code
|
||||
to display OpenCV images on another computer with a display.
|
||||
For example, a headless Raspberry Pi with no display can run OpenCV code
|
||||
and can display OpenCV images on a Mac with a display.
|
||||
USAGE:
|
||||
First, start this "display server" program on the computer with a display:
|
||||
# imageShowServer.py
|
||||
import zmqimage
|
||||
zmq = zmqimage.zmqImageShowServer()
|
||||
print "Starting zmqImageShow Server..."
|
||||
print " press Ctrl-C to stop"
|
||||
while True: # Until Ctrl-C is pressed, will repeatedly
|
||||
zmq.imshow() # display images sent from the headless computer
|
||||
Run the above program by:
|
||||
python imageShowServer.py
|
||||
Leave the above program running in its own terminal window.
|
||||
Then, run a program like the one below on the headless computer.
|
||||
In most cases, it will be run using ssh into the headless computer
|
||||
from another terminal window on the computer with a display.
|
||||
The python lines below represent a program fragment as an example.
|
||||
Use zmq.imshow("Image Display Name", image) instead of
|
||||
cv2.imshow("Image Display Name", image) and the images will
|
||||
display on the computer running the program above:
|
||||
import numpy as np
|
||||
import cv2
|
||||
import zmqimage
|
||||
print "Connecting to zmqShowImage Server ... "
|
||||
zmq = zmqimage.zmqConnect()
|
||||
image = np.zeros((500, 500), dtype="uint8")
|
||||
zmq.imshow("Zero Image 500 x 500", image)
|
||||
# build a rectangular mask & display it
|
||||
mask = np.zeros(image.shape[:2], dtype="uint8")
|
||||
cv2.rectangle(mask, (0, 90), (300, 450), 255, -1)
|
||||
zmq.imshow("Rectangular Mask", mask)
|
||||
CAVEATS:
|
||||
There is no error checking and only Ctrl-C stops the display server.
|
||||
While zmq.imshow() works like cv2.imshow(), no other
|
||||
cv2 display functions are implemented.
|
||||
Uses tcp style sockets; sockets and tcp addresses have
|
||||
defaults in the classes but may be overridden.
|
||||
AUTHOR:
|
||||
Jeff Bass, https://github.com/jeffbass, jeff@yin-yang-ranch.com
|
||||
"""
|
||||
|
||||
import zmq
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
|
||||
class SerializingSocket(zmq.Socket):
|
||||
"""A class with some extra serialization methods
|
||||
|
||||
send_array sends numpy arrays with metadata necessary
|
||||
for reconstructing the array on the other side (dtype,shape).
|
||||
Also sends array name for display with cv2.show(image).
|
||||
recv_array receives dict(arrayname,dtype,shape) and an array
|
||||
and reconstructs the array with the correct shape and array name.
|
||||
"""
|
||||
|
||||
def send_array(self, A, arrayname="NoName", flags=0, copy=True, track=False):
|
||||
"""send a numpy array with metadata and array name"""
|
||||
md = dict(
|
||||
arrayname=arrayname,
|
||||
dtype=str(A.dtype),
|
||||
shape=A.shape,
|
||||
)
|
||||
self.send_json(md, flags | zmq.SNDMORE)
|
||||
return self.send(A, flags, copy=copy, track=track)
|
||||
|
||||
def recv_array(self, flags=0, copy=True, track=False):
|
||||
"""recv a numpy array, including arrayname, dtype and shape"""
|
||||
md = self.recv_json(flags=flags)
|
||||
msg = self.recv(flags=flags, copy=copy, track=track)
|
||||
A = np.frombuffer(msg, dtype=md["dtype"])
|
||||
return (md["arrayname"], A.reshape(md["shape"]))
|
||||
|
||||
|
||||
class SerializingContext(zmq.Context):
|
||||
_socket_class = SerializingSocket
|
||||
|
||||
|
||||
class zmqConnect:
|
||||
"""A class that opens a zmq REQ socket on the headless computer"""
|
||||
|
||||
def __init__(self, connect_to="tcp://jeff-mac:5555"):
|
||||
"""initialize zmq socket for sending images to display on remote computer"""
|
||||
"""connect_to is the tcp address:port of the display computer"""
|
||||
self.zmq_context = SerializingContext()
|
||||
self.zmq_socket = self.zmq_context.socket(zmq.REQ)
|
||||
self.zmq_socket.connect(connect_to)
|
||||
|
||||
def imshow(self, arrayname, array):
|
||||
"""send image to display on remote server"""
|
||||
if array.flags["C_CONTIGUOUS"]:
|
||||
# if array is already contiguous in memory just send it
|
||||
self.zmq_socket.send_array(array, arrayname, copy=False)
|
||||
else:
|
||||
# else make it contiguous before sending
|
||||
array = np.ascontiguousarray(array)
|
||||
self.zmq_socket.send_array(array, arrayname, copy=False)
|
||||
message = self.zmq_socket.recv()
|
||||
|
||||
|
||||
class zmqImageShowServer:
|
||||
"""A class that opens a zmq REP socket on the display computer to receive images"""
|
||||
|
||||
def __init__(self, open_port="tcp://*:5555"):
|
||||
"""initialize zmq socket on viewing computer that will display images"""
|
||||
self.zmq_context = SerializingContext()
|
||||
self.zmq_socket = self.zmq_context.socket(zmq.REP)
|
||||
self.zmq_socket.bind(open_port)
|
||||
|
||||
def imshow(self, copy=False):
|
||||
"""receive and show image on viewing computer display"""
|
||||
arrayname, image = self.zmq_socket.recv_array(copy=False)
|
||||
# print "Received Array Named: ", arrayname
|
||||
# print "Array size: ", image.shape
|
||||
cv2.imshow(arrayname, image)
|
||||
cv2.waitKey(0)
|
||||
self.zmq_socket.send(b"OK")
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -22,28 +22,23 @@ import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import torch
|
||||
import umsgpack
|
||||
from omegaconf import DictConfig
|
||||
from pyrad.cameras.rays import RayBundle
|
||||
from pyrad.utils.config import ViewerConfig
|
||||
|
||||
import pyrad.viewer.backend.cameras as c
|
||||
import pyrad.viewer.backend.geometry as g
|
||||
import pyrad.viewer.backend.transformations as tf
|
||||
import pyrad.viewer.server.cameras as c
|
||||
import pyrad.viewer.server.geometry as g
|
||||
import pyrad.viewer.server.transformations as tf
|
||||
from pyrad.cameras.cameras import Camera, get_camera, get_intrinsics_from_intrinsics_matrix
|
||||
from pyrad.cameras.rays import RayBundle
|
||||
from pyrad.utils import profiler
|
||||
from pyrad.viewer.backend import ViewerWindow, Visualizer
|
||||
from pyrad.viewer.backend.utils import get_intrinsics_matrix_and_camera_to_world_h
|
||||
from pyrad.utils.config import ViewerConfig
|
||||
from pyrad.viewer.server import Viewer
|
||||
from pyrad.viewer.server.utils import get_intrinsics_matrix_and_camera_to_world_h
|
||||
|
||||
|
||||
class CameraChangeException(Exception):
|
||||
"""Basic camera exception to interrupt visualizer"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class SetTrace:
|
||||
"""Basic trace function"""
|
||||
@@ -64,11 +59,12 @@ class VisualizerState:
|
||||
|
||||
def __init__(self, config: ViewerConfig):
|
||||
self.config = config
|
||||
|
||||
self.vis = None
|
||||
if self.config.enable:
|
||||
window = ViewerWindow(zmq_url=self.config.zmq_url)
|
||||
self.vis = Visualizer(window=window)
|
||||
logging.info("Connected to viewer at %s", self.config.zmq_url)
|
||||
zmq_url = self.config.zmq_url
|
||||
self.vis = Viewer(zmq_url=zmq_url)
|
||||
logging.info("Connected to viewer at %s", zmq_url)
|
||||
self.vis.delete()
|
||||
else:
|
||||
logging.info("Continuing without viewer.")
|
||||
@@ -130,8 +126,9 @@ class VisualizerState:
|
||||
self.check_done_render = False
|
||||
while not self.check_done_render:
|
||||
data = self.vis["/Cameras/Main Camera"].get_object()
|
||||
message = umsgpack.unpackb(data)
|
||||
camera_object = message["object"]["object"]
|
||||
if data is None:
|
||||
return
|
||||
camera_object = data["object"]["object"]
|
||||
if self.prev_camera_matrix is None or not np.array_equal(camera_object["matrix"], self.prev_camera_matrix):
|
||||
with self.lock:
|
||||
self.check_interrupt_vis = True
|
||||
@@ -155,8 +152,9 @@ class VisualizerState:
|
||||
The image is sent of a TCP connection and then uses WebRTC to send it to the viewer.
|
||||
"""
|
||||
data = self.vis["/Cameras/Main Camera"].get_object()
|
||||
message = umsgpack.unpackb(data)
|
||||
camera_object = message["object"]["object"]
|
||||
if data is None:
|
||||
return
|
||||
camera_object = data["object"]["object"]
|
||||
# hacky way to prevent overflow check to see if < 100; TODO(make less hacky)
|
||||
if self.prev_camera_matrix is not None and np.array_equal(camera_object["matrix"], self.prev_camera_matrix):
|
||||
self.res_upscale_factor = min(self.res_upscale_factor * 2, 100)
|
||||
@@ -193,17 +191,16 @@ class VisualizerState:
|
||||
if outputs is not None:
|
||||
# gross hack to get the image key, depending on which keys the graph uses
|
||||
rgb_key = "rgb" if "rgb" in outputs else "rgb_fine"
|
||||
# TODO: make it such that the TCP connection doesn't need float64
|
||||
image = outputs[rgb_key].cpu().numpy().astype("float64") * 255
|
||||
image = (outputs[rgb_key].cpu().numpy() * 255).astype("uint8")
|
||||
self.vis["/Cameras/Main Camera"].set_image(image)
|
||||
return outputs
|
||||
|
||||
|
||||
def get_vis(zmq_url="tcp://0.0.0.0:6000"):
|
||||
"""Returns the vis object."""
|
||||
window = ViewerWindow(zmq_url)
|
||||
vis = Visualizer(window=window)
|
||||
return vis
|
||||
def get_default_vis():
|
||||
"""Returns the default Visualizer."""
|
||||
zmq_url = "tcp://0.0.0.0:6000"
|
||||
viewer = Viewer(zmq_url=zmq_url)
|
||||
return viewer
|
||||
|
||||
|
||||
def show_box_test(vis):
|
||||
@@ -211,12 +208,6 @@ def show_box_test(vis):
|
||||
vis["box"].set_object(g.Box([1.0, 1.0, 1.0]), material=g.MeshPhongMaterial(color=0xFF0000))
|
||||
|
||||
|
||||
def get_random_color():
|
||||
color = np.random.rand(3) * 255.0
|
||||
color = tuple([int(x) for x in color])
|
||||
return color
|
||||
|
||||
|
||||
def show_ply(vis, ply_path, name="ply", color=None):
|
||||
"""Show the PLY file in the 3D viewer. Specify the full filename as input."""
|
||||
assert ply_path.endswith(".ply")
|
||||
@@ -15,15 +15,20 @@
|
||||
from __future__ import absolute_import, division, print_function
|
||||
|
||||
import numpy as np
|
||||
import msgpack
|
||||
import msgpack_numpy
|
||||
import umsgpack
|
||||
import zmq
|
||||
|
||||
from .commands import Delete, SetObject, GetObject, SetImage, SetProperty, SetTransform
|
||||
from pyrad.viewer.server.socket import SerializingContext
|
||||
|
||||
from .commands import Delete, SetObject, GetObject, SetProperty, SetTransform
|
||||
from .path import Path
|
||||
|
||||
|
||||
class ViewerWindow(object):
|
||||
context = zmq.Context()
|
||||
# context = SerializingContext()
|
||||
|
||||
def __init__(self, zmq_url):
|
||||
self.zmq_url = zmq_url
|
||||
@@ -42,28 +47,53 @@ class ViewerWindow(object):
|
||||
return self.client.recv()
|
||||
|
||||
|
||||
class Visualizer(object):
|
||||
def __init__(self, window):
|
||||
self.window = window
|
||||
self.path = Path(("pyrad",)) # TODO(ethan): change this
|
||||
class Viewer(object):
|
||||
"""Visualizer class for connecting to the bridge server."""
|
||||
|
||||
def __init__(self, zmq_url: str = None, window: ViewerWindow = None):
|
||||
if zmq_url is None and window is None:
|
||||
raise ValueError("Must specify either zmq_url or window.")
|
||||
if window is None:
|
||||
self.window = ViewerWindow(zmq_url=zmq_url)
|
||||
else:
|
||||
self.window = window
|
||||
self.path = Path(("pyrad",))
|
||||
|
||||
@staticmethod
|
||||
def view_into(window, path):
|
||||
vis = Visualizer(window)
|
||||
def view_into(window: ViewerWindow, path: Path):
|
||||
"""Returns a new Viewer but keeping the same ViewerWindow."""
|
||||
vis = Viewer(window=window)
|
||||
vis.path = path
|
||||
return vis
|
||||
|
||||
def __getitem__(self, path):
|
||||
return Visualizer.view_into(self.window, self.path.append(path))
|
||||
return Viewer.view_into(self.window, self.path.append(path))
|
||||
|
||||
def set_object(self, geometry, material=None):
|
||||
return self.window.send(SetObject(geometry, material, self.path))
|
||||
|
||||
def get_object(self):
|
||||
return self.window.send(GetObject(self.path))
|
||||
"""Get the object at the current path."""
|
||||
data = self.window.send(GetObject(self.path))
|
||||
data = umsgpack.unpackb(data)
|
||||
if isinstance(data, str) and data.find("error") == 0:
|
||||
# some error meaning that the object does not exist
|
||||
return None
|
||||
return data
|
||||
|
||||
def set_image(self, image):
|
||||
return self.window.send(SetImage(image, self.path))
|
||||
"""Set the image"""
|
||||
type_ = "set_image"
|
||||
path = self.path.lower()
|
||||
data = msgpack.packb(image, default=msgpack_numpy.encode, use_bin_type=True)
|
||||
self.window.client.send_multipart(
|
||||
[
|
||||
type_.encode("utf-8"),
|
||||
path.encode("utf-8"),
|
||||
data,
|
||||
]
|
||||
)
|
||||
return self.window.client.recv()
|
||||
|
||||
def set_transform(self, matrix=np.eye(4)):
|
||||
assert matrix.shape == (4, 4)
|
||||
@@ -76,4 +106,4 @@ class Visualizer(object):
|
||||
return self.window.send(Delete(self.path))
|
||||
|
||||
def __repr__(self):
|
||||
return "<Visualizer using: {window} at path: {path}>".format(window=self.window, path=self.path)
|
||||
return "<Viewer using: {window} at path: {path}>".format(window=self.window, path=self.path)
|
||||
@@ -0,0 +1,20 @@
|
||||
import argparse
|
||||
from pyrad.viewer.server.server import ZMQWebSocketBridge
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
parser = argparse.ArgumentParser(description="Listen for ZeroMQ commands")
|
||||
parser.add_argument("--zmq-url", "-z", type=str, nargs="?", default=None)
|
||||
parser.add_argument("--websocket-port", "-wp", type=str, nargs="?", default=None)
|
||||
args = parser.parse_args()
|
||||
bridge = ZMQWebSocketBridge(zmq_url=args.zmq_url, websocket_port=args.websocket_port)
|
||||
print(bridge)
|
||||
try:
|
||||
bridge.run()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -11,13 +11,13 @@ import numpy as np
|
||||
import umsgpack
|
||||
from tqdm import tqdm
|
||||
|
||||
import pyrad.viewer.backend.geometry as g
|
||||
import pyrad.viewer.backend.transformations as tf
|
||||
import pyrad.viewer.server.geometry as g
|
||||
import pyrad.viewer.server.transformations as tf
|
||||
from pyrad.cameras.camera_paths import InterpolatedCameraPath
|
||||
from pyrad.cameras.cameras import get_camera
|
||||
from pyrad.data.utils import get_dataset_inputs
|
||||
from pyrad.utils.io import get_absolute_path
|
||||
from pyrad.viewer.backend.vis_utils import get_vis, set_camera
|
||||
from pyrad.viewer.server.vis_utils import get_vis, set_camera
|
||||
|
||||
|
||||
def test_drawing():
|
||||
|
||||
Reference in new issue
Block a user