6a7c8820b8
* Copy in `viser` communication infrastructure * Update background image * Add build directory to fix viser HTTP server May want to remove these later * Copy in `viser` communication infrastructure * Update background image * Pass dataset vis using viser * initial impl of ViewerParameter * idk what i did but brent and matt know * Suppress eslint typescript warnings * implement ViewerElement base class, then ViewerButton, ViewerSlider, etc * client -> server message setup * multi folder stuff * Stream controls over viser * import * add types to viewer_param.py * Move another message to viser * Update folder color * Use non throttled messages * multiple folders works * Cleanup, more porting * starting impl of control panel in viser * More porting * scenebox to viser * Cleanup * Fix render updates * More cleanup, docs, typing * start control_panel.py * set_hidden logic but not merged in * control_panel replicates behavior of existing control panel (visually), except for setting the list of output options. not yet integrated with viewer_utils.py for logic flow. also implemented set_disabled and set_hidden in javscript/ViewerElement side * Remove some original leva components * add some train util logic, attempt set_value, doesn't work on javascript side fully yet (value updates but not slider visually * bye zmq * missing call to decoder * 'gui_set' works properly, control panel now replicates behavior of original * remove print * Fix status viewer message * More cleanup * support set_options in dropdown * hiding buttons works correctly * fix set_options, switch from cur_value to get_value() * Fix slowdown * output options * colormap logic converted * add handle to cb_hook * fix some lambda bugs * cleanup * Save checkpoint logic readded * Cleanup and minor fixes * snappier ui * Refactoring * Fix state update bugs * Bug fixes * Write less to redux * added comments to reducer code * Render state machine (#1727) * state machine works, but train step throttling not implemented * more step logic, cleanup * remove random try catcH * mutex logic for render * Update lock logic * types and docs * added some state machine logic to help rerenders * merge --------- Co-authored-by: Justin <justin.g.kerr@gmail.com> * Refactor and cleanup * More cleanup and docs * docstring to gui_utils * docstrings, rename viewer_param.py to viewer_elements.py * Fix viewer, remove bridge stuff * Render on connect * Fix lint errors * More eslint fixes * render_freq logic * better rerender logic for crop * More cleanup * Purge * Cleanup * Lint fixes * Fix cache * better parsing logic * Fix infinite test * More cleanup * typo * Update webdocs * Update docs * rerender on custom hooks * add more docs * Use shared utilities in `viser.infra` * rework callback to use elements instead of handles * Fix ESlint errors * change naming of handle to element, add setter for value * cleanup --------- Co-authored-by: Matt Tancik <mattt4001@gmail.com> Co-authored-by: Justin <justin.g.kerr@gmail.com> Co-authored-by: Ethan Weber <eweb0124@gmail.com> Co-authored-by: Matthew Tancik <tancik@berkeley.edu>
7.4 KiB
:og:description: Nerfstudio Documentation
:og:image: https://assets.nerf.studio/opg.png
:width: 400
:align: center
:alt: nerfstudio
:class: only-light
:width: 400
:align: center
:alt: nerfstudio
:class: only-dark
Nerfstudio provides a simple API that allows for a simplified end-to-end process of creating, training, and visualizing NeRFs. The library supports an interpretable implementation of NeRFs by modularizing each component. With modular NeRF components, we hope to create a user-friendly experience in exploring the technology. Nerfstudio is a contributor-friendly repo with the goal of building a community where users can easily build upon each other's contributions.
It's as simple as plug and play with nerfstudio!
On top of our API, we are committed to providing learning resources to help you understand the basics of (if you're just getting started), and keep up-to-date with (if you're a seasoned veteran) all things NeRF. As researchers, we know just how hard it is to get onboarded with this next-gen technology. So we're here to help with tutorials, documentation, and more!
Finally, have feature requests? Want to add your brand-spankin'-new NeRF model? Have a new dataset? We welcome any and all contributions! Please do not hesitate to reach out to the nerfstudio team with any questions via Discord.
We hope nerfstudio enables you to build faster 🔨 learn together 📚 and contribute to our NeRF community 💖.
Contents
:hidden:
:caption: Getting Started
quickstart/installation
quickstart/first_nerf
quickstart/custom_dataset
quickstart/viewer_quickstart
quickstart/export_geometry
quickstart/data_conventions
Contributing<reference/contributing>
:hidden:
:caption: Extensions
extensions/blender_addon
extensions/unreal_engine
extensions/sdfstudio
:hidden:
:caption: NeRFology
nerfology/methods/index
nerfology/model_components/index
:hidden:
:caption: Developer Guides
developer_guides/new_methods
developer_guides/pipelines/index
developer_guides/viewer/index
developer_guides/config
developer_guides/debugging_tools/index
:hidden:
:caption: Reference
reference/cli/index
reference/api/index
This documentation is organized into 3 parts:
- 🏃♀️ Getting Started: a great place to start if you are new to nerfstudio. Contains a quick tour, installation, and an overview of the core structures that will allow you to get up and running with nerfstudio.
- 🧪 Nerfology: want to learn more about the tech itself? We're here to help with our educational guides. We've provided some interactive notebooks that walk you through what each component is all about.
- 🤓 Developer Guides: describe all of the components and additional support we provide to help you construct, train, and debug your NeRFs. Learn how to set up a model pipeline, use the viewer, create a custom config, and more.
- 📚 Reference: describes each class and function. Develop a better understanding of the core of our technology and terminology. This section includes descriptions of each module and component in the codebase.
Supported Methods
- Nerfacto: our de facto NeRF method combines modules focused on quality with modules focused on faster rendering. Nerfstudio easily lets us experiment with the best of both worlds!
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
- Instant NGP: Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
- Mipnerf: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
- NerfW: Neural Radiance Fields for Unconstrained Photo Collections
- Semantic NeRF: In-Place Scene Labelling and Understanding with Implicit Scene Representation
We'll be constantly growing this list! So make sure to check back in to see our updates.
Eager to contribute? We'd love to see you use nerfstudio in implementing new (or even existing) methods! Feel free to contact us directly or view our Contributor's Guide to see how you can get your model on this list!
Quicklinks
| Github | Official Github Repo |
| Discord | Join Discord Community |
| Viewer | Web-based Nerf Viewer |
How-to Videos
| Using the Viewer | Demo video on how to run nerfstudio and use the viewer. |
| Using Record3D | Demo video on how to run nerfstudio without using COLMAP. |
Built On
:width: 150
:alt: tyro
:class: only-light
:target: https://github.com/brentyi/tyro
:width: 150
:alt: tyro
:class: only-dark
:target: https://github.com/brentyi/tyro
- Easy to use config system
- Developed by Brent Yi
:width: 250
:alt: tyro
:class: only-light
:target: https://github.com/KAIR-BAIR/nerfacc
:width: 250
:alt: tyro
:class: only-dark
:target: https://github.com/KAIR-BAIR/nerfacc
- Library for accelerating NeRF renders
- Developed by Ruilong Li
Citation
You can find a paper writeup of the framework on arXiv.
If you use this library or find the documentation useful for your research, please consider citing:
@article{nerfstudio,
author = {Tancik, Matthew and Weber, Ethan and Ng, Evonne and Li, Ruilong and Yi,
Brent and Kerr, Justin and Wang, Terrance and Kristoffersen, Alexander and Austin,
Jake and Salahi, Kamyar and Ahuja, Abhik and McAllister, David and Kanazawa, Angjoo},
title = {Nerfstudio: A Modular Framework for Neural Radiance Field Development},
journal = {arXiv preprint arXiv:2302.04264},
year = {2023},
}