nerfstudio/docs/notebooks/visualize_samples.ipynb

Sample Representation

In [1]:

import torch
from pyrad.cameras import cameras
from pyrad.utils import plotly as vis
import plotly.graph_objects as go

A core component of volumetric rendering is computing samples along a ray. The representation of these samples is important. We use conical frustums to represent samples. The conical frustums are defined via the following parameters:

In [2]:

cx = 2
cy = 2
fx = 10
fy = 10

num_samples = 3
near_plane = 1
far_plane = 3

camera = cameras.PinholeCamera(cx, cy, fx, fy)
ray_bundle = camera.get_camera_ray_bundle()

bins = torch.linspace(near_plane, far_plane, num_samples + 1)[..., None]
ray_samples = ray_bundle.get_ray_samples(bin_starts=bins[:-1, :], bin_ends=bins[1:, :])

vis_rays = vis.get_ray_bundle_lines(ray_bundle, color="teal", length=far_plane)

fig = go.Figure(
    data=[vis_rays] + vis.get_frustums_mesh_list(ray_samples.frustums),
    layout=go.Layout(scene_camera=dict(up=dict(x=0, y=1, z=0)), title="Frustums"),
)
fig.show()

These frustums can be converted into point samples using frustum.get_positions().

In [3]:

fig = go.Figure(
    data=[vis_rays, vis.get_frustum_points(ray_samples.frustums)],
    layout=go.Layout(scene_camera=dict(up=dict(x=0, y=1, z=0)), title="Frustums as Points"),
)
fig.show()

We can also appoximate the frustums using Gaussians with frustum.get_gaussian_blob().

In [4]:

fig = go.Figure(
    data=[vis_rays] + vis.get_gaussian_ellipsoids_list(ray_samples.frustums.flatten().get_gaussian_blob()),
    layout=go.Layout(scene_camera=dict(up=dict(x=0, y=1, z=0)), title="Frustums as Gaussians"),
)
fig.show()