nerfstudio/docs/tutorials/pipelines/dataparsers.ipynb

DataParsers

:align: center

What is a DataParser?

The dataparser returns DataparserOutputs, which puts all the various datasets into a common format.

@dataclass
class DataparserOutputs:
    """Dataset inputs for the image dataset and the ray generator."""

    image_filenames: List[Path]
    """Filenames for the images."""
    cameras: Cameras
    """Camera object storing collection of camera information in dataset"""
    alpha_color: Optional[TensorType[3]] = None
    """color of dataset background"""
    scene_box: SceneBox = SceneBox()
    """scene box of dataset. Could be used to bound the scene, or to provide the scene scale depending on model."""
    ...

@dataclass
class DataParser:

    @abstractmethod
    def _generate_dataset_inputs(self, split: str = "train") -> DataparserOutputs:
        """Abstract method that returns the dataparser outputs for the given split.

        Args:
            split: Which dataset split to generate data for (train/eval/test).

        Returns:
            DataparserOutputs for the specified dataset and split.
        """

Example

Here is an example where we implement a DataParser for our Nerfstudio data format.

@dataclass
class NerfstudioDataParserConfig(DataParserConfig):
    """Nerfstudio dataset config"""

    _target: Type = field(default_factory=lambda: Nerfstudio)
    """Target class to instantiate."""
    data_directory: Path = Path("data/nerfstudio/poster")
    """Directory specifying location of data."""
    scale_factor: float = 1.0
    """How much to scale the camera origins by."""
    downscale_factor: int = 1
    """How much to downscale images."""
    scene_scale: float = 1.0
    """How much to scale the region of interest by."""
    orientation_method: Literal["pca", "up"] = "up"
    """The method to use for orientation."""
    train_split_percentage: float = 0.9
    """The percent of images to use for training. The remaining images are for eval.
    """

@dataclass
class Nerfstudio(DataParser):
    """Nerfstudio Dataset"""

    config: NerfstudioDataParserConfig

    def _generate_dataset_inputs(self, split="train") -> DataParserOutputs:
      """Returns DataParserOutputs."""

Our Implementations

Nerfstudio

This is our custom dataparser.

:color: primary
:outline:
See the code!

Blender

We support the synthetic Blender dataset from the original NeRF paper.

:color: primary
:outline:
See the code!

Instant NGP

This supports the Instant NGP dataset.

:color: primary
:outline:
See the code!

MipNeRF 360

:color: primary
:outline:
See the code!

Record3D

This dataparser can use recorded data from a >= iPhone 12 Pro using the Record3D app. Record a video and export with the EXR + JPG sequence format. Unzip export and rgb folder before training.

:color: primary
:outline:
See the code!