mirror of
https://github.com/nerfstudio-project/nerfstudio.git
synced 2026-11-14 11:54:01 +08:00
b17268e83b
* starting the framework for dreamfusion * Removed view direction from RGB to calculate albedo * Started diffusion * Added dreamfusion to CLI, removed direction embeddings * Normals in dreamfusion * Lambertian shading added * Train normals in Dreamfusion * Stable diffusion from Stable Dreamfusion * Fix normal orientation loss * Configurable light direction sampling * adding density initalization at origin * V1 datamanager * Don't train normals for first 1k iters * SDS loss added to dreamfusion * Dreamfusion pipeline * Background MLP and SphereCollider * Normals prediction training fixed * Sigmoid on background, prompt in config * Shading hyperparameter adjustements * Shading hyperparameter tuning * Location based prompting, density initialization fades, LR change * Cleaned imports, more config options * Tuning, more config changes * remove unused sampler, normalize predicted normals from network * change position encoding to direction encoding for background color mlp * Updating dreamfusion to use custom trainer, fixed bug, multi-gpu support (put diffusion on one specific device), merged with main * Hard coding less, offloading to config * Appeasing the linter * First pass opacity / sparsity losses * Small patches and configs * Accidentally pushed logs * Removing logs for real this time * Transmittance loss bug fix * 12 GB Dreamfusion! * Lint * Remove unecissary collider changes * Start SD cleanup * Cleanup SD code * adding overhead location prompting * Use lambertian renderer * turn grad accum off by default, decrease background mlp size, add overhead text embedding * Cleanup * Fix normals vis * change uniform sampler to prediction network, add option to jit unet for faster sds_loss * Use shaders * cleaning up jit code * add warning if using non jit unet * add proposal network optimizer to config * speed up render and cleanup code * Clean up datamanager * Add distortion loss * fix wanb * adding interpolated location sampling * Update configs * interpolated location prompting option, lowering init density, transmittance loss interpolation flipped * rename alphas mult to alphas loss mult * Mess around with normals * add option to use random bg color + bg mlp * Remove Dreamfusion Trainer (#1423) * Dreamfusion trainer no longer needed * Added typing for grad scaler * SDS return loss * Fix CFG * Cleanup * Update configs * Fix opacity loss, use unform sampling first round of prop * add prompt edditing to viewer * Choose your own diffusion * remove dreamfusion pipeline file * merging + remove dreamfusion pipeline file * Update configs * add stable diff utils * cleanup * cleanup * Fix test * positional prompting bugfix * Warmups * tune * lint * Remove dataparser * merging + text embedding bugfix * Set data path to launch viewer for DreamFusion (#1776) * Set data path to launch viewer for DreamFusion * Apply formatter * Fallback to 'base_dir' if data manager has no path * Make if statement non-inline * Reformat --------- Co-authored-by: Matthew Tancik <mattt4001@gmail.com> * add deepfloyd option first try * merging * merging from main * merging * removing comments, cleaning code, making stable diff utils usable for deepfloyd, remove deepfloyd text encoder after use * undo chagnes in data folder * cleaning code, removing unneccesary changes from main, removed normals prediction network * remove old scripts files, rename generative pipeline flag to no_input_data, revert to default histogram padding in proposal sampler * changing pipeline flag * move no input data flag from pipeline to datamanager * undo config and merge in deepfloyd from main * merging grad scaler pipeline addition * merging + setting proposal net scheduler * revert nerfacto config change * typing + linting * rename dreamfusion datamanager to random camras datamanager, check datamanager typing in pipeline and trainer * fix test import * fix test import * changing datamanager check in trainer, add typing in random cameras datamanager * add Tensor to randomcamerasdatamanager * remove old comments * resolve tyro warning * update fields to use tcnn * lint * add packages for deepfloyd * add versions * docs first draft * rename to generfacto, clean docs * fix import error * minor changes * fix orientation loss bug, retune params * cleanup * typing * trying docs fix * fixing generative package imports * fix pyright * docs fix * merge w main * docs minor change * tuning + bugfixes * remove torch.backends --------- Co-authored-by: Abhik Ahuja <ahujaabhik@berkeley.edu> Co-authored-by: David McAllister <davidrmc2@gmail.com> Co-authored-by: Jake Austin <jake-austin@berkeley.edu> Co-authored-by: Matthew Tancik <tancik@berkeley.edu> Co-authored-by: Matt Tancik <mattt4001@gmail.com> Co-authored-by: SeungWoo Yoo <dreamy1534@gmail.com> Co-authored-by: ayaanzhaque <ayaanzhaque@gmail.com> Co-authored-by: Ayaan Haque <ayaanzhaque@berkeley.edu>
69 lines
2.1 KiB
Markdown
69 lines
2.1 KiB
Markdown
# Methods
|
|
|
|
We provide a set of pre implemented nerfstudio methods.
|
|
|
|
**The goal of nerfstudio is to modularize the various NeRF techniques as much as possible.**
|
|
|
|
As a result, many of the techniques from these pre-implemented methods can be mixed 🎨.
|
|
|
|
## Running a method
|
|
|
|
It's easy!
|
|
|
|
```bash
|
|
ns-train {METHOD_NAME}
|
|
```
|
|
|
|
To list the available methods run:
|
|
|
|
```bash
|
|
ns-train --help
|
|
```
|
|
|
|
## Implemented Methods
|
|
|
|
The following methods are supported in nerfstudio:
|
|
|
|
```{toctree}
|
|
:maxdepth: 1
|
|
Instant-NGP<instant_ngp.md>
|
|
Instruct-NeRF2NeRF<in2n.md>
|
|
K-Planes<kplanes.md>
|
|
LERF<lerf.md>
|
|
Mip-NeRF<mipnerf.md>
|
|
NeRF<nerf.md>
|
|
Nerfacto<nerfacto.md>
|
|
NeRFPlayer<nerfplayer.md>
|
|
Tetra-NeRF<tetranerf.md>
|
|
TensoRF<tensorf.md>
|
|
Generfacto<generfacto.md>
|
|
```
|
|
|
|
(own_method_docs)=
|
|
|
|
## Adding Your Own Method
|
|
|
|
If you're a researcher looking to develop new NeRF-related methods, we hope that you find nerfstudio to be a useful tool. We've provided documentation about integrating with the nerfstudio codebase, which you can find [here](../../developer_guides/new_methods.md).
|
|
|
|
We also welcome additions to the list of methods above. To do this, simply create a pull request with the following changes,
|
|
|
|
1. Add a markdown file describing the model to the `docs/nerfology/methods` folder
|
|
2. Update the above list of implement methods in this file.
|
|
3. Add the method to the {ref}`this<third_party_methods>` list in `docs/index.md`.
|
|
4. Add a new `ExternalMethod` entry to the `nerfstudio/configs/external_methods.py` file.
|
|
|
|
For the method description, please refer to the [Instruct-NeRF2NeRF](in2n) page as an example of the layout. Please try to include the following information:
|
|
|
|
- Installation instructions
|
|
- Instructions for running the method
|
|
- Requirements (dataset, GPU, ect)
|
|
- Method description (the more detailed the better, treat it like a blog post)
|
|
|
|
You are welcome to include assets (such as images or video) in the description, but please host them elsewhere.
|
|
|
|
:::{admonition} Note
|
|
:class: note
|
|
|
|
Please ensure that the documentation is clear and easy to understand for other users who may want to try out your method.
|
|
:::
|