mirror of
https://github.com/nerfstudio-project/nerfstudio.git
synced 2026-11-14 11:54:01 +08:00
Add support for Project Aria datasets. (#2617)
* Add support for Project Aria datasets. 1) An export script for processing Project Aria datasets 2) Add support for Fisheye624 cameras 3) Add support for center-crop masking and sampling Example commands for running nerfstudio on Project Aria data: python scripts/datasets/process_project_aria.py --vrs_file=<path_to_vrs> --mps_data_dir=<path to MPS directory> --output_dir=<nerfstudio_data_dir> ns-train nerfacto --data <nerfstudio_data_dir> nerfstudio-data --orientation-method none * Clean up type errors * Fix formatting * Fix linter errors, import, and use absolute file paths * Add aria subcommand to ns-process-data * Appease ruff * Add Aria to docs * Fix docs, add back import try/except * nit: path.as_posix() -> str(path) --------- Co-authored-by: Brent Yi <yibrenth@gmail.com>
This commit is contained in:
1 parent
01d57b90dc
commit
6ccbc2de45
10 files changed
+634
-40
No files matched your search
@@ -10,7 +10,7 @@ ns-process-data {video,images,polycam,record3d} --data {DATA_PATH} --output-dir
|
||||
|
||||
A full set of arguments can be found {doc}`here</reference/cli/ns_process_data>`.
|
||||
|
||||
We Currently support the following custom data types:
|
||||
We currently support the following custom data types:
|
||||
| Data | Capture Device | Requirements | `ns-process-data` Speed |
|
||||
| ----------------------------- | -------------- | ----------------------------------------------- | ----------------------- |
|
||||
| 📷 [Images](images_and_video) | Any | [COLMAP](https://colmap.github.io/install.html) | 🐢 |
|
||||
@@ -22,6 +22,7 @@ We Currently support the following custom data types:
|
||||
| 🖥 [Metashape](metashape) | Any | [Metashape](https://www.agisoft.com/) | 🐇 |
|
||||
| 🖥 [RealityCapture](realitycapture) | Any | [RealityCapture](https://www.capturingreality.com/realitycapture) | 🐇 |
|
||||
| 🖥 [ODM](odm) | Any | [ODM](https://github.com/OpenDroneMap/ODM) | 🐇 |
|
||||
| 👓 [Aria](aria) | Aria glasses | [Project Aria](https://projectaria.com/) | 🐇 |
|
||||
|
||||
(images_and_video)=
|
||||
|
||||
@@ -348,6 +349,31 @@ ns-process-data odm --data /path/to/dataset --output-dir {output directory}
|
||||
ns-train nerfacto --data {output directory}
|
||||
```
|
||||
|
||||
(aria)=
|
||||
|
||||
## Aria
|
||||
|
||||
1. Install projectaria_tools:
|
||||
|
||||
```bash
|
||||
conda activate nerfstudio
|
||||
pip install projectaria-tools'[all]'
|
||||
```
|
||||
|
||||
2. Download a VRS file from Project Aria glasses, and run Machine Perception Services to extract poses.
|
||||
|
||||
3. Convert to nerfstudio format.
|
||||
|
||||
```bash
|
||||
ns-process-data aria --vrs-file /path/to/vrs/file --mps-data-dir /path/to/mps/data --output-dir {output directory}
|
||||
```
|
||||
|
||||
4. Train!
|
||||
|
||||
```bash
|
||||
ns-train nerfacto --data {output directory}
|
||||
```
|
||||
|
||||
(360_data)=
|
||||
|
||||
## 360 Data (Equirectangular)
|
||||
@@ -459,4 +485,4 @@ If the depth of the scene is unviewable and looks too close or expanded when vie
|
||||
- The IPD can be modified in the `cameras.py` script as the variable `vr_ipd` (default is 64 mm).
|
||||
- Compositing with Blender Objects and VR180 or ODS Renders
|
||||
- Configure the Blender camera as panoramic and equirectangular. For the VR180 Blender camera, set the panoramic longitude min and max to -90 and 90.
|
||||
- Change the Stereoscopy mode to "Parallel" set the Interocular Distance to 0.064 m.
|
||||
- Change the Stereoscopy mode to "Parallel" set the Interocular Distance to 0.064 m.
|
||||
Reference in new issue
Block a user