merging main

This commit is contained in:
terrance committed 2023-05-10 21:26:04 +00:00
commit 80cdb73406
221 files changed
+50645 -2513

No files matched your search

+40 -7
View File
@@ -5,7 +5,7 @@ Training model on existing datasets is only so fun. If you would like to train o
To process your own data run:
```bash
ns-process-data {video,images,polycam,insta360,record3d} --data {DATA_PATH} --output-dir {PROCESSED_DATA_DIR}
ns-process-data {video,images,polycam,record3d} --data {DATA_PATH} --output-dir {PROCESSED_DATA_DIR}
```
A full set of arguments can be found {doc}`here</reference/cli/ns_process_data>`.
@@ -20,6 +20,7 @@ We Currently support the following custom data types:
| 📱 [KIRI Engine](kiri) | IOS or Android | [KIRI Engine App](https://www.kiriengine.com/) | 🐇 |
| 📱 [Record3D](record3d) | IOS with LiDAR | [Record3D app](https://record3d.app/) | 🐇 |
| 🖥 [Metashape](metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 |
| 🖥 [RealityCapture](realitycapture) | Any | [RealityCapture](https://www.capturingreality.com/realitycapture) | 🐇 |
(images_and_video)=
@@ -33,12 +34,6 @@ To assist running on custom data we have a script that will process a video or f
- COLMAP can be finicky. Try your best to capture overlapping, non-blurry images.
:::
:::{admonition} Note
:class: warning
- If COLMAP opens in a new window, simply close it to resume processing.
:::
### Processing Data
```bash
@@ -264,6 +259,8 @@ ns-train nerfacto --data {output directory}
## Metashape
All images must use the same sensor type (but multiple sensors are supported).
1. Align your images using Metashape. `File -> Workflow -> Align Photos...`
```{image} https://user-images.githubusercontent.com/3310961/203389662-12760210-2b52-49d4-ab21-4f23bfa4a2b3.png
@@ -292,6 +289,29 @@ ns-process-data metashape --data {data directory} --xml {xml file} --output-dir
ns-train nerfacto --data {output directory}
```
<<<<<<< HEAD
=======
(realitycapture)=
## RealityCapture
1. Align your images using RealityCapture. `ALIGNMENT -> Align Images`
2. Export the camera alignment as a `csv` file. Choose `Internal/External camera parameters`
3. Convert the data to the nerfstudio format.
```bash
ns-process-data realitycapture --data {data directory} --csv {csv file} --output-dir {output directory}
```
4. Train with nerfstudio!
```bash
ns-train nerfacto --data {output directory}
```
>>>>>>> b1ddb9e695b789fae47e0f760a5c7f59f25af0bb
(360_data)=
## 360 Data (Equirectangular)
@@ -303,10 +323,17 @@ Equirectangular data is data that has been taken by a 360 camera such as Insta36
For a set of equirectangular images, process the data using the following command:
```bash
<<<<<<< HEAD
ns-process-data images --camera-type equirectangular --images-per-equirect {8, or 14} --data {data directory} --output-dir {output directory}
```
The images-per-equirect argument is the number of images that will be sampled from each equirectangular image. We have found that 8 images per equirectangular image is sufficient for most use cases so it defaults to that. However, if you find that there isn't enough detail in the nerf or that colmap is having trouble aligning the images, you can try increasing the number of images per equirectangular image to 14.
=======
ns-process-data images --camera-type equirectangular --images-per-equirect {8, or 14} --crop-factor {top bottom left right} --data {data directory} --output-dir {output directory}
```
The images-per-equirect argument is the number of images that will be sampled from each equirectangular image. We have found that 8 images per equirectangular image is sufficient for most use cases so it defaults to that. However, if you find that there isn't enough detail in the nerf or that colmap is having trouble aligning the images, you can try increasing the number of images per equirectangular image to 14. See the video section below for details on cropping.
>>>>>>> b1ddb9e695b789fae47e0f760a5c7f59f25af0bb
### Videos
@@ -326,4 +353,10 @@ The `crop-factor` argument is optional but often very helpful. This is because e
<img src="imgs/equirect_crop.jpg">
<<<<<<< HEAD
This obscene mesh of human is obviously not helpful in training a nerf so we can remove it by cropping the bottom 20% of the image. This can be done by using the `--crop-factor 0 0.2 0 0` argument.
=======
The pixels representing the distorted hand and head are obviously not useful in training a nerf so we can remove it by cropping the bottom 20% of the image. This can be done by using the `--crop-factor 0 0.2 0 0` argument.
If cropping only needs to be done from the bottom, you can use the `--crop-bottom [num]` argument which would be the same as doing `--crop-factor 0.0 [num] 0.0 0.0`
>>>>>>> b1ddb9e695b789fae47e0f760a5c7f59f25af0bb