Metric3D v2
Metric3D v2 (Wei Yin and Mu Hu et al.): metric depth and surface normals from one image, given the focal length. Commercially usable; the authors ask commercial users to contact them.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
metric3d-vit-large | Middlebury 2014 half (994x1482) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 629 438 200 158 149 91.4 | 3077 / 3122 3275 / 3350 1484 / 1696 1516 / 1688 1126 / 1148 1010 (engine) |
Details
- Original code: https://github.com/YvanYin/Metric3D at
eb5b6fa - Code licence: BSD-2-Clause (permissive); files:
LICENSE - Weights licence: BSD-2-Clause (assumed from code)
- Licence evidence: https://github.com/YvanYin/Metric3D/blob/eb5b6fac0dc155e4e52f576e304fbf11655ff339/README.md#license-and-contact
- Training data: 16M images from 18 public metric-depth datasets (indoor, outdoor, driving) per the Metric3D v2 paper (arXiv 2404.15506); original asks commercial users to contact the authors
- Licence note: The upstream README says 'for further commercial inquiries, please contact' the authors; no weights licence is stated, so the BSD-2-Clause code licence is assumed.
- Kernel libraries:
fastbb - Variants:
metric3d-vit-large - Outputs: depth, normals, confidence; kind:
metric - Input size: any H x W >= 8, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
metric3d-vit-large3000x4000 (2.3 GiB peak allocated, 2.9 reserved) metric3d-vit-large994x1482, first call: portable + torch.compile 55 s (cold Inductor cache), TensorRT fp16 97 s (ONNX + engine).metric3d-vit-largeoriginal 994x1482: torch.autocast(fp16) around the forwardmetric3d-vit-largeload / first call: original 9.96 s / 0.876 s, portable 3.45 s / 0.233 s, kernels 0.31† s / 0.159 s- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
metric3d-vit-large | NYUv2 test, AbsRel / delta1 | 0.0547 / 0.9673 | 0.0547 / 0.9673 | 0.0547 / 0.9673 | 0.0546 / 0.9673 | |
metric3d-vit-large | iBims-1, AbsRel / delta1 | 0.0334 / 0.9855 | 0.0334 / 0.9855 | 0.0334 / 0.9855 | 0.0334 / 0.9855 | |
metric3d-vit-large | ETH3D two-view train left, AbsRel / delta1 | 0.0740 / 0.9411 | 0.0740 / 0.9411 | 0.0740 / 0.9411 | 0.0741 / 0.9410 | possibly in the training data |
metric3d-vit-large | Middlebury 2014 trainingH left, AbsRel / delta1 | 0.0247 / 0.9961 | 0.0247 / 0.9961 | 0.0247 / 0.9961 | 0.0247 / 0.9961 | possibly in the training data |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1 % and p99 <= 3 % of the reference's median |value| (absolute error of the output depth).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
metric3d-vit-large | 0.00028 / 0.00075 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120 |
Limitations
- The normals convention (camera axes / sign) is not checked against a dataset; it is passed through as original returns it.