Models · Monocular depth

LingBot-Depth

LingBot-Depth (Robbyant, 2026): metric depth completion from an image plus an incomplete or noisy metric depth map (RGB-D camera, LiDAR), and a depth-completion variant (dc). Apache-2.0 code and weights.

experimental Fork of robbyant/lingbot-depth@f3a237e · backends: portable, kernels

Checkpoint

Paper 2026 Added in 0.2.0 Evaluated with 0.2.0

Kernels, RTX 5060 Ti 16 GB 161 ms 994x1482, fp16
vs the faster original 1.8× original bf16: 297 ms
Peak memory 1,453 MiB original: 2,267 MiB
Accuracy vs original fp32 ±0.0000 kernels met_absrel, NYUv2 test (every 4th: 164 images) with a 192x256 depth prompt (0.0053 original)

Use it

pip install fastforks

Lean install with only what this model needs: pip install fastforks-core

from fastforks import depth

model = depth.load("lingbot-depth")  # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.depth  # also: mask
result.backend  # what ran: mode, device, dtype, kernels
fastforks bench lingbot-depth

Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.

Speed and memory

Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.

Input: synthetic, 994x1482 (the task’s standard size)

BackenddtypeTime per call (ms)vs original fp16Peak memory
allocated / reserved
Load (s)First call (s)
Original fp32fp32not measured ––––
Original (own default)the authors' mixed-precision option (bfloat16); upstream MDMModel.infer (resolution_level 9, apply_mask off), no xFormersbf16 297–2,267 MiB/2,338 MiB3.50.42
Portable fp32fp32not measured ––––
Portablefp16 2801.1×1,559 MiB/1,872 MiB1.40.53
Kernelsfp16 1611.8×1,453 MiB/1,590 MiB1.40.33
Portable + torch.compile 2511.2×1,381 MiB/1,572 MiB28 (build)–
TensorRT fp16fp16 1881.6×1,126 MiBenginenot comparable66 (build)–

Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-15 to 2026-11-16. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.

Other devices

DeviceBackenddtypems / callPeak memoryOutput vs referenceInput
Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-17Portablefp163376 3,228 MiB processskip994x1482
Portablefp323363 4,011 MiB processskip994x1482
Kernelsfp16not implemented–994x1482

CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.

Licence and weights

Commercial use allowed

Code
Apache-2.0 LICENSE
Weights
Apache-2.0 source
Training data
Large-scale real and synthetic RGB-D data with masked depth modelling per the tech report

Our reading of the licences, not legal advice.

Weights

lingbot-depth-pretrain-vitl-14-v0.5/model.pt 1.3 GB· hosted on Hugging Face

fastforks downloads and checks these files on first use (fastforks download lingbot-depth).

Accuracy

The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.

NYUv2 test (every 4th: 164 images) with a 192x256 depth prompt

AbsRel / delta1 (metric)

Rundtypemet_absrel ↑met_d1 ↑Δ met_absrel vs fp32 originalimagesms / image
Original fp32fp320.00531.0000 reference 164 1236
Original fp16bf160.00561.0000 +0.0003 164 306
Portable fp32fp320.00531.0000 ±0.0000 164 1228
Portablefp160.00531.0000 ±0.0000 164 289
Kernelsfp160.00531.0000 ±0.0000 164 167

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Prompt depth models: the prompt is a 192x256 low-resolution depth built from the ground truth (mean of the valid pixels per cell, holes filled with the nearest cell), the output is metric depth compared without any scale fit; AbsRel / delta1 over the pixels with ground truth, mean over images. The prompt comes from the ground truth, so the numbers are not comparable with the paper's (ARKitScenes / ScanNet++ with a LiDAR prompt, L1 in mm): they compare our modes with the original. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-16; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 30c7a8e9, 2026-11-18; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 68b2f345, 2026-11-16.

iBims-1 (100 images) with a 192x256 depth prompt

AbsRel / delta1 (metric)

Rundtypemet_absrel ↑met_d1 ↑Δ met_absrel vs fp32 originalimagesms / image
Original fp32fp320.00890.9951 reference 100 1224
Original fp16bf160.00920.9950 +0.0003 100 303
Portable fp32not measured
Portablefp160.00890.9951 ±0.0000 100 286
Kernelsfp160.00890.9951 ±0.0000 100 166

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Prompt depth models: the prompt is a 192x256 low-resolution depth built from the ground truth (mean of the valid pixels per cell, holes filled with the nearest cell), the output is metric depth compared without any scale fit; AbsRel / delta1 over the pixels with ground truth, mean over images. The prompt comes from the ground truth, so the numbers are not comparable with the paper's (ARKitScenes / ScanNet++ with a LiDAR prompt, L1 in mm): they compare our modes with the original. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-16; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 68b2f345, 2026-11-16.

Closeness to the original

Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.005 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.5 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).

CheckToleranceMeasured
CPU fp32, max |diff|≤ 0.005 (relative)not recorded
GPU default dtypes, mean |diff|≤ 0.015 (relative)0.00046 (relative)
GPU default dtypes, p99 |diff|≤ 0.05 (relative)0.00254 (relative)

Recorded for lingbot-depth on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-14, with these kernels: dpt_fusedfastbb

Details

LingBot-Depth (Robbyant, 2026): metric depth completion from an image and an incomplete or noisy metric depth map (RGB-D); a DINOv2 ViT-L that attends over image and depth tokens, with MoGe's conv neck and depth / mask heads.

Input / output
Input: one RGB image. Output: a depth or inverse-depth map.
What the numbers mean, and which models need the camera focal length

kind (a card field, result.kind) says what the numbers mean: metric = metres; scale_invariant, affine_invariant, affine_invariant_log (log depth) and affine_invariant_inverse are relative (unknown scale or shift, not metres). Metric models that use the focal length take model(image, intrinsics=K) (K [3, 3], pixels of the input): metric3d-vit-large and da3metric-large need it for correct metres (da3metric-large: metres = output x focal / 300; without intrinsics= a 300 px focal length is assumed, wrong for a real camera). unidepth-v2-vitl14, unik3d-vitl, depth-pro and MoGe estimate the camera themselves (given intrinsics= replace fx); zoedepth-nk and the relative models (da3mono-large too) take none. Normals-only models (kind none) return result.normals [B, H, W, 3] (unit vectors, camera frame: x right, y down, z forward).

Input size
any H x W >= 1, resized or padded inside the model (ViT patch 14); tested up to 3000x4000 (1,692 MiB allocated)
Options
  • resolution_level=90 to 9· changes the output
Outputs
depth, mask
Checkpoint
exp_output=false
CUDA kernels
  • fastbbfast backbone kernels: flash attention, window attention, instance norm and rotary embeddings for ViT and similar backbones
  • dpt_fusedfused fp16 matrix multiplies and convolutions for DINOv2 backbones and DPT depth heads
Apple (MPS)
fp32, fp16

Limitations

  • The depth map is required: call the model with `prompt_depth=` a metric map in metres (any size, holes as 0 / NaN / inf). Its holes change the ViT's token count, so the kernels mode runs the ViT per image. Export keeps every depth token (masked) and has a fixed size.
Full reference, generated from the repository

LingBot-Depth (Robbyant, 2026): metric depth completion from an image plus an incomplete or noisy metric depth map (RGB-D camera, LiDAR), and a depth-completion variant (dc). Apache-2.0 code and weights.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
lingbot-depthsynthetic (994x1482)original
portable
kernels
portable + torch.compile
TensorRT fp16
297
280
161
251
188
2267 / 2338
1559 / 1872
1453 / 1590
1381 / 1572
1821 / 1876
lingbot-depth-dcsynthetic (994x1482)original
portable
kernels
portable + torch.compile
TensorRT fp16
297
281
161
252
188
2267 / 2338
1559 / 1872
1453 / 1590
1381 / 1572
1821 / 1876
Details
  • Original code: https://github.com/robbyant/lingbot-depth at f3a237e
  • Code licence: Apache-2.0 (permissive); files: LICENSE
  • Weights licence: Apache-2.0
  • Licence evidence: https://github.com/robbyant/lingbot-depth/blob/main/LICENSE and the Hugging Face model cards (license: apache-2.0)
  • Training data: Large-scale real and synthetic RGB-D data with masked depth modelling per the tech report
  • Options: resolution_level = 9 (0 to 9)
  • Kernel libraries: fastbb, dpt_fused
  • Variants: lingbot-depth, lingbot-depth-dc
  • Outputs: depth, mask; kind: metric
  • Input size: any H x W >= 1, resized or padded inside the model (ViT patch 14)
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): lingbot-depth 3000x4000 (1.7 GiB peak allocated, 2.1 reserved); lingbot-depth-dc 3000x4000 (1.7 GiB peak allocated, 2.1 reserved)
  • lingbot-depth 994x1482, first call: portable + torch.compile 28 s (cold Inductor cache), TensorRT fp16 66 s (ONNX + engine).
  • lingbot-depth-dc 994x1482, first call: portable + torch.compile 32 s (cold Inductor cache), TensorRT fp16 67 s (ONNX + engine).
  • lingbot-depth original 994x1482: the authors' mixed-precision option (bfloat16); upstream MDMModel.infer (resolution_level 9, apply_mask off), no xFormers
  • lingbot-depth-dc original 994x1482: the authors' mixed-precision option (bfloat16); upstream MDMModel.infer (resolution_level 9, apply_mask off), no xFormers
  • lingbot-depth load / first call: original 4.81 s / 1.377 s, portable 1.38 s / 0.531 s, kernels 1.36 s / 0.332 s
  • lingbot-depth-dc load / first call: original 7.54 s / 1.364 s, portable 1.29 s / 0.532 s, kernels 1.33 s / 0.343 s
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5

Measured

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Prompt depth models: the prompt is a 192x256 low-resolution depth built from the ground truth (mean of the valid pixels per cell, holes filled with the nearest cell), the output is metric depth compared without any scale fit; AbsRel / delta1 over the pixels with ground truth, mean over images. The prompt comes from the ground truth, so the numbers are not comparable with the paper's (ARKitScenes / ScanNet++ with a LiDAR prompt, L1 in mm): they compare our modes with the original. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portableportable fp32kernelsnote
lingbot-depthNYUv2 test (every 4th: 164 images) with a 192x256 depth prompt, AbsRel / delta1 (metric)0.0053 / 1.00000.0056 / 1.00000.0053 / 1.00000.0053 / 1.00000.0053 / 1.0000
lingbot-depthiBims-1 (100 images) with a 192x256 depth prompt, AbsRel / delta1 (metric)0.0089 / 0.99510.0092 / 0.99500.0089 / 0.99510.0089 / 0.9951
lingbot-depth-dcNYUv2 test (every 4th: 164 images) with a 192x256 depth prompt, AbsRel / delta1 (metric)0.0084 / 0.9998

Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 68b2f345, 2026-11-16; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 30c7a8e9, 2026-11-18; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 68b2f345, 2026-11-16.

Parity

CPU, float32: max |diff| <= 0.005 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.5 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).

variantkernels error vs original, mean / p99GPU, torchkernels used
lingbot-depth0.00046 / 0.00254RTX 5060 Ti, torch 2.14.1+cu130dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120
lingbot-depth-dc0.00058 / 0.00365RTX 5060 Ti, torch 2.14.1+cu130dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120

Limitations

  • The depth map is required: call the model with prompt_depth= a metric map in metres (any size, holes as 0 / NaN / inf). Its holes change the ViT's token count, so the kernels mode runs the ViT per image. Export keeps every depth token (masked) and has a fixed size.

Example output

No example rendered for this checkpoint yet.