Models · Monocular depth

Marigold V2

Relative depth from one photo by a very large image-generation model run for a single step: accurate but slow and memory-hungry (about 2 GB of GPU memory, 9 GB of host RAM).

experimental Fork of huawei-bayerlab/marigold-v2@main · backends: portable, kernels

Checkpoint marigold-v2-depth

Paper 2026 Added in 0.2.0 Evaluated with 0.2.0

Kernels, RTX 5060 Ti 16 GB 3886 ms 994x1482, bf16
vs the faster original – original not timed
Peak memory 1,498 MiB original not measured
Accuracy vs original fp32 – original not run on the test set

Use it

pip install fastforks

Lean install with only what this model needs: pip install fastforks-core

from fastforks import depth

model = depth.load("marigold-v2-depth")  # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.depth
result.backend  # what ran: mode, device, dtype, kernels
fastforks bench marigold-v2-depth

Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.

Speed and memory

Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs bf16. Portable runs bf16 on CUDA.

BackenddtypeTime per call (ms)vs originalPeak memory
allocated / reserved
Load (s)First call (s)
Original fp32fp32not measured ––––
Original fp16fp16out of memory––––
Portable fp32fp32not measured ––––
Portablebf16 5093–1,408 MiB/1,698 MiB7.85.09
Kernelsbf16 3886–1,498 MiB/1,780 MiB6.23.89
Portable + torch.compiletimeout after 900 s (audit 2026-11-16, RTX 5060 Ti: compile fails––––
TensorRTplanned for 0.2.1fails––––

Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-15. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.

Other devices

DeviceBackenddtypems / callPeak memoryOutput vs referenceInput
Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-17Portablefp32cannot run here–994x1482
Kernelsbf16not implemented–994x1482

CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.

Licence and weights

Commercial use allowed

Code
Apache-2.0 LICENSENOTICE
Weights
Apache-2.0 source
Training data
Qwen-Image-Edit-2509 (Alibaba, Apache-2.0) fine-tuned (LoRA + VAE decoder) on synthetic depth data per the Marigold V2 paper (arXiv 2609.08084); the training data terms are not checked
  • The converted weight file (NF4 transformer, LoRA, folded constants, 2-D VAE) is hosted by fastforks under the originals' Apache-2.0 licence; NOTICE keeps both attributions and lists the changes.

Our reading of the licences, not legal advice.

Weights

marigold-v2-depth-log-stage2-nf4.safetensors 9.0 GB· hosted on Hugging Face

fastforks downloads and checks these files on first use (fastforks download marigold-v2-depth).

Accuracy

The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.

NYUv2 test

AbsRel / delta1

RundtypeAbsRel ↓δ1 ↑imagesms / image
Original fp32not measured
Original fp16bf16out of memory
Portable fp32fp320.03860.9840 654 1328
Portablebf160.03860.9840 654 1310
Kernelsbf160.03860.9840 654 777

The original code was not run on this dataset, so the table cannot show the distance to it.

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the authors’ code, same checkpoint and protocol. Measured on original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-16; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6d3bab64, 2026-11-17; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14.

iBims-1

AbsRel / delta1

RundtypeAbsRel ↓δ1 ↑imagesms / image
Original fp32not measured
Original fp16not measured
Portable fp32fp320.02910.9877 100 1384
Portablebf160.02910.9877 100 1362
Kernelsbf160.02910.9877 100 820

The original code was not run on this dataset, so the table cannot show the distance to it.

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Measured on portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6d3bab64, 2026-11-17; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14.

Closeness to the original

Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.02 vs the original. GPU, bfloat16 (the default), portable and kernels: mean |diff| <= 1 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).

CheckToleranceMeasured
CPU fp32, max |diff|≤ 0.02 (relative)not recorded
GPU default dtypes, mean |diff|≤ 0.01 (relative)0 (relative)
GPU default dtypes, p99 |diff|≤ 0.05 (relative)0 (relative)

Recorded for marigold-v2-depth on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-14, with these kernels: nf4

Details

Marigold V2 (Pavlovic, Wandel, Obukhov et al. 2026): affine-invariant log depth by one rectified-flow step of Qwen-Image-Edit-2509's 20 B transformer (NF4 4-bit, as the authors run it) with a rank-128 depth LoRA and a fine-tuned Qwen VAE decoder (depth/Log-stage2).

Input / output
Input: one RGB image. Output: a depth or inverse-depth map.
What the numbers mean, and which models need the camera focal length

kind (a card field, result.kind) says what the numbers mean: metric = metres; scale_invariant, affine_invariant, affine_invariant_log (log depth) and affine_invariant_inverse are relative (unknown scale or shift, not metres). Metric models that use the focal length take model(image, intrinsics=K) (K [3, 3], pixels of the input): metric3d-vit-large and da3metric-large need it for correct metres (da3metric-large: metres = output x focal / 300; without intrinsics= a 300 px focal length is assumed, wrong for a real camera). unidepth-v2-vitl14, unik3d-vitl, depth-pro and MoGe estimate the camera themselves (given intrinsics= replace fx); zoedepth-nk and the relative models (da3mono-large too) take none. Normals-only models (kind none) return result.normals [B, H, W, 3] (unit vectors, camera frame: x right, y down, z forward).

Input size
any H x W >= 1, resized or padded inside the model; tested up to 1988x2880 (12.8 GiB allocated)
Options
  • seed=2026· changes the output
  • long_side=0>= 0· changes the output
  • exact_mask=false· changes the output
  • stream=true· changes the output
Outputs
depth
CUDA kernels
  • nf4

Limitations

  • 20 B transformer in 4 bit: 8.4 GB of weights; by default (`stream=True`, CUDA) it runs in ~1-4 GB of device memory from 8.4 GB of page-locked host RAM (load fails with an error when less is free); `stream=False` keeps it resident (about 9-11 GB of device memory, 16 GB cards: up to ~1024 px).
  • Diffusion model: the VAE posterior sample comes from a CPU generator seeded with `seed` and differs from the original code's (unseeded, device) sample.
  • The enlargement to a multiple of 16 is bicubic (the original: OpenCV Lanczos); inputs with sides that are multiples of 16 are not resized.
  • The output is log depth up to scale and shift (`kind = "affine_invariant_log"`): align in log space, not with the other families' depth outputs.
  • A traced or exported module reuses the noise of its traced shape; CPU inference is possible but slow (bf16 GEMMs of a 20 B model).
Full reference, generated from the repository

Relative depth from one photo by a very large image-generation model run for a single step: accurate but slow and memory-hungry (about 2 GB of GPU memory, 9 GB of host RAM).

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
marigold-v2-depthsynthetic (994x1482)original
portable
kernels
portable + torch.compile
TensorRT fp16
OOM
5093
3886
fails: timeout after 900 s (audit 2026-11-16, RTX 5060 Ti: compile
planned for 0.2.1
–
1408 / 1698
1498 / 1780
–
–

Weight streaming

Warm inference time and peak reserved GPU memory for the same input with both settings. Streamed weights use pinned host memory.

ModelModeInputGPUPyTorchResident (stream=False)Streamed (stream=True)
marigold-v2-depthkernels518x770RTX 5060 Ti2.14.1+cu1301,071 ms / 9,482 MiB1,087 ms / 1,238 MiB
marigold-v2-depthkernels994x1482RTX 5060 Ti2.14.1+cu1303,868 ms / 10,548 MiB3,887 ms / 2,106 MiB
marigold-v2-depthportable518x770RTX 5060 Ti2.14.1+cu1301,666 ms / 9,582 MiB1,683 ms / 1,190 MiB
marigold-v2-depthportable994x1482RTX 5060 Ti2.14.1+cu1305,071 ms / 10,250 MiB5,092 ms / 2,106 MiB
Details
  • Original code: https://github.com/huawei-bayerlab/marigold-v2 at main
  • Code licence: Apache-2.0 (permissive); files: LICENSE, NOTICE
  • Weights licence: Apache-2.0
  • Licence evidence: https://huggingface.co/huawei-bayerlab/marigold-v2/blob/main/README.md (license: apache-2.0; base model Qwen/Qwen-Image-Edit-2509)
  • Training data: Qwen-Image-Edit-2509 (Alibaba, Apache-2.0) fine-tuned (LoRA + VAE decoder) on synthetic depth data per the Marigold V2 paper (arXiv 2609.08084); the training data terms are not checked
  • Licence note: The converted weight file (NF4 transformer, LoRA, folded constants, 2-D VAE) is hosted by fastforks under the originals' Apache-2.0 licence; NOTICE keeps both attributions and lists the changes.
  • Options: seed = 2026, long_side = 0 (>= 0), exact_mask = False, stream = True
  • Kernel libraries: nf4
  • Variants: marigold-v2-depth
  • Not in this release (planned for 0.2.1): marigold-v2-depth ONNX / TensorRT export: the NF4 transformer expands its weights per call; no ONNX path yet
  • Outputs: depth; kind: affine_invariant_log
  • Input size: any H x W >= 1, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): marigold-v2-depth 1988x2880 (12.8 GiB peak allocated, 13.5 reserved)
  • marigold-v2-depth load / first call: portable 7.8 s / 5.094 s, kernels 6.2 s / 3.887 s
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5

Measured

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal halfportableportable fp32kernelsnote
marigold-v2-depthNYUv2 test, AbsRel / delta1out of memory0.0386 / 0.98400.0386 / 0.98400.0386 / 0.9840
marigold-v2-depthiBims-1, AbsRel / delta10.0291 / 0.98770.0291 / 0.98770.0291 / 0.9877

Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6d3bab64, 2026-11-17; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14.

Parity

CPU, float32: max |diff| <= 0.02 vs the original. GPU, bfloat16 (the default), portable and kernels: mean |diff| <= 1 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).

variantkernels error vs original, mean / p99GPU, torchkernels used
marigold-v2-depth0.0 / 0.0RTX 5060 Ti, torch 2.14.1+cu130nf4@e236d80ba78b:sm_120

Limitations

  • 20 B transformer in 4 bit: 8.4 GB of weights; by default (stream=True, CUDA) it runs in ~1-4 GB of device memory from 8.4 GB of page-locked host RAM (load fails with an error when less is free); stream=False keeps it resident (about 9-11 GB of device memory, 16 GB cards: up to ~1024 px).
  • Diffusion model: the VAE posterior sample comes from a CPU generator seeded with seed and differs from the original code's (unseeded, device) sample.
  • The enlargement to a multiple of 16 is bicubic (the original: OpenCV Lanczos); inputs with sides that are multiples of 16 are not resized.
  • The output is log depth up to scale and shift (kind = "affine_invariant_log"): align in log space, not with the other families' depth outputs.
  • A traced or exported module reuses the noise of its traced shape; CPU inference is possible but slow (bf16 GEMMs of a 20 B model).

Example output

No example rendered for this checkpoint yet.