Pose optimization (#721)

* pose opt default nerfacto.

* added SO3 cam pose optimization which is slightly faster (~2.2m/iter) than BARF

* integrating record3d

* fix linting errors

* fix save checkpoint logic

* added record3d data

* workaround to avoid flag

* moved dict

* black

* record3d docs

* re-add typing to exp_map.

* Pose optimization refactor

* Fix camera optimizer param group logic

* Update docs + comments

* Update docs, shorten names

* Appease pylint

* repeat -> tile, apply missing name change

* Fix optimizer as scheduler bug

* Fix test

Co-authored-by: akristoffersen <akristoffersen@berkeley.edu>
Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com>
Co-authored-by: Brent Yi <yibrenth@gmail.com>
This commit is contained in:
authored and GitHub committed 2022-10-07 10:11:33 -07:00
1 parent 2531242d1d
commit cac177717f
8 files changed
+230 -103

No files matched your search

+15 -1
View File
@@ -119,5 +119,19 @@ Record a video and export with the EXR + JPG sequence format.
Then, move the exported capture folder from your iPhone to your computer.
```
ns-train nerfacto record3d-data --data {RECORD3D_CAPTURE_DIR/EXR_RGBD}
ns-train nerfacto --pipeline.datamanager.train-camera-optimizer.mode SO3xR3 record3d-data --data {RECORD3D_CAPTURE_DIR/EXR_RGBD}
```
You can try with our data to see the correct formatting.
```shell
ns-download-data --dataset record3d
```
This will download data to `data/record3d/bear`. Then you can train, and don't forget to open up the viewer.
```python
ns-train nerfacto --pipeline.datamanager.train-camera-optimizer.mode SO3xR3 record3d-data --data data/record3d/bear
```
Notice we are adding the option `--pipeline.datamanager.train-camera-optimizer.mode SO3xR3`, which is experimental but turns on camera pose optimization for higher quality results with record3d. This is because the camera poses from record3d are quite noisy, unlike those from COLMAP (from the `ns-process-data` script).