Depth supervision for Nerfacto (#1173)

* Implement depth supervision

* Fixes

* Change normalize dtype

* refactor

* Change depth weight

* Small refactor

* Change docstring

* Removu unused import

* Add assumption to depth comment

* Implement URF loss

* Remove default depth loss type param

* URF fixes

* Fix depth loss scale

* Visualize gt depths

* Fixes & documentation

* Documentation and formatting

* Fix formatting problems

* Fix linter problems

* Fix tests

* Fix tests
This commit is contained in:
Michal Pandy authored and GitHub committed 2023-01-02 10:13:38 +01:00
1 parent fa6f279751
commit 3da877050d
12 files changed
+460 -4

No files matched your search

+4
View File
@@ -67,3 +67,7 @@ For a transform matrix, the first 3 columns are the +X, +Y, and +Z defining the
]
}
```
**Depth images**
To train with depth supervision, you can also provide a `depth_file_path` for each frame in your `transforms.json` and use one of the methods that support additional depth losses (e.g., depth-nerfacto). The depths are assumed to be 16-bit or 32-bit and to be in millimeters to remain consistent with [Polyform](https://github.com/PolyCam/polyform). You can adjust this scaling factor using the `depth_unit_scale_factor` parameter in `NerfstudioDataParserConfig`. Note that by default, we resize the depth images to match the shape of the RGB images.