mirror of
https://github.com/nerfstudio-project/nerfstudio.git
synced 2026-11-18 13:53:44 +08:00
NeuS Implementation (#1444)
* add initial method configs and data manager * add neus scheduler * add initial surface model and neus * add Neus ray sampler * add sdf field * move scale/shift calc to math utils * add debug and fix sdf_dataset typo * update vscode neus debug * fix dataset and get initial working version * add magic number from original repo * black fixes * clean up get weights/transmittance from alphas * clean up typos in sdf datamanager * type annotations on neus scheduler and remove unused params * clean up and annotate sdf field along w/ other fixes * more clean up * remove unnessary directions norm * resolve comments * update neus scheduler to follow refactor * fix linter errors * fix some typos * remove get_weights_from_alpha * more comment fixes * more linter error fixes * clean up and add comments * more little fixes for types/comments * more comments * rename NeuS Scheduler to DelayedScheduler * rename DelayedScheduler to CosineDecayScheduler
This commit is contained in:
1 parent
1b18b44d81
commit
ddfa463f4d
16 files changed
+1376
-53
No files matched your search
@@ -311,3 +311,44 @@ def masked_reduction(
|
||||
input_tensor[valid] = input_tensor[valid] / mask[valid]
|
||||
input_tensor = torch.mean(input_tensor)
|
||||
return input_tensor
|
||||
|
||||
|
||||
def normalized_depth_scale_and_shift(
|
||||
prediction: TensorType[1, 32, "mult"], target: TensorType[1, 32, "mult"], mask: TensorType[1, 32, "mult"]
|
||||
):
|
||||
"""
|
||||
More info here: https://arxiv.org/pdf/2206.00665.pdf supplementary section A2 Depth Consistency Loss
|
||||
This function computes scale/shift required to normalizes predicted depth map,
|
||||
to allow for using normalized depth maps as input from monocular depth estimation networks.
|
||||
These networks are trained such that they predict normalized depth maps.
|
||||
|
||||
Solves for scale/shift using a least squares approach with a closed form solution:
|
||||
Based on:
|
||||
https://github.com/autonomousvision/monosdf/blob/d9619e948bf3d85c6adec1a643f679e2e8e84d4b/code/model/loss.py#L7
|
||||
Args:
|
||||
prediction: predicted depth map
|
||||
target: ground truth depth map
|
||||
mask: mask of valid pixels
|
||||
Returns:
|
||||
scale and shift for depth prediction
|
||||
"""
|
||||
# system matrix: A = [[a_00, a_01], [a_10, a_11]]
|
||||
a_00 = torch.sum(mask * prediction * prediction, (1, 2))
|
||||
a_01 = torch.sum(mask * prediction, (1, 2))
|
||||
a_11 = torch.sum(mask, (1, 2))
|
||||
|
||||
# right hand side: b = [b_0, b_1]
|
||||
b_0 = torch.sum(mask * prediction * target, (1, 2))
|
||||
b_1 = torch.sum(mask * target, (1, 2))
|
||||
|
||||
# solution: x = A^-1 . b = [[a_11, -a_01], [-a_10, a_00]] / (a_00 * a_11 - a_01 * a_10) . b
|
||||
scale = torch.zeros_like(b_0)
|
||||
shift = torch.zeros_like(b_1)
|
||||
|
||||
det = a_00 * a_11 - a_01 * a_01
|
||||
valid = det.nonzero()
|
||||
|
||||
scale[valid] = (a_11[valid] * b_0[valid] - a_01[valid] * b_1[valid]) / det[valid]
|
||||
shift[valid] = (-a_01[valid] * b_0[valid] + a_00[valid] * b_1[valid]) / det[valid]
|
||||
|
||||
return scale, shift
|
||||
Reference in new issue
Block a user