Models · Monocular depth

PointDiT

Google's PointDiT: a diffusion transformer that denoises a 3D point map in three steps from one image, conditioned on frozen DINOv3 features. Code and weights are Apache-2.0; the DINOv3 encoder it loads is under the DINOv3 License.

experimental Fork of google-research/pointdit@11f53a3 · backends: portable, kernels

Checkpoint

Paper 2026 Added in 0.2.0 Evaluated with 0.2.0

Kernels, RTX 5060 Ti 16 GB 20.5 ms 994x1482, fp16
vs the faster original 2.4× original bf16: 48.7 ms
Peak memory 462 MiB original: 1,214 MiB
Accuracy vs original fp32 ±0.0000 kernels AbsRel, NYUv2 test (0.0453 original)

Use it

pip install fastforks

Lean install with only what this model needs: pip install fastforks-core

from fastforks import depth

model = depth.load("pointdit-b-512")  # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.points  # also: depth
result.backend  # what ran: mode, device, dtype, kernels
fastforks bench pointdit-b-512

Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.

Speed and memory

Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.

BackenddtypeTime per call (ms)vs original fp16Peak memory
allocated / reserved
Load (s)First call (s)
Original fp32upstream wild-image resize (~1024 tokens), 3 Euler stepsfp32 1050.46×977 MiB/1,006 MiB4.90.28
Original (own default)the authors' mixed-precision option (bfloat16); upstream wild-image resize (~1024 tokens), 3 Euler steps; authors bf16 autocastbf16 48.7–1,214 MiB/1,246 MiB2.60.11
Portable fp32fp32not measured ––––
Portablefp16 31.11.6×446 MiB/504 MiB1.70.03
Kernelsfp16 20.52.4×462 MiB/630 MiB1.70.02
Portable + torch.compile 23.72.1×465 MiB/524 MiB34 (build)–
TensorRT fp16fp16 15.53.1×394 MiBenginenot comparable87 (build)–

Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-15 to 2026-11-16. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.

Other devices

DeviceBackenddtypems / callPeak memoryOutput vs referenceInput
Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-15Originalfp32468 3,167 MiB process–994x1482
Portablefp16337 1,912 MiB processmatches994x1482
Portablefp32367 1,812 MiB processmatches994x1482
Kernelsfp16not implemented–994x1482

CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.

Licence and weights

Commercial use allowed

Code
Apache-2.0 LICENSE
Weights
LicenseRef-DINOv3-License source
Training data
Mixture of synthetic and real point-map datasets ("mixdata") per the PointDiT paper; the DINOv3 encoder: LVD-1689M (not released)
  • The PointDiT code and weights are Apache-2.0; the frozen DINOv3 ViT-B/16 and ViT-L/16 encoders are under the DINOv3 License (Meta: attribution and conditions apply; gated on Hugging Face); read it before commercial use.

Our reading of the licences, not legal advice.

Weights

dinov3-vitb16-pretrain-lvd1689m/model.safetensors 343 MB· hosted on Hugging Face
pointditb-512-mixdata-nodinov3-1d42aacf.pth 1.1 GB· hosted on Hugging Face

fastforks downloads and checks these files on first use (fastforks download pointdit-b-512).

Accuracy

The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.

NYUv2 test

AbsRel / delta1

RundtypeAbsRel ↓δ1 ↑Δ AbsRel vs fp32 originalimagesms / image
Original fp32fp320.04530.9773 reference 654 109
Original fp16bf160.04540.9773 −0.0001 654 48.6
Portable fp32fp320.04530.9773 ±0.0000 654 103
Portablefp160.04530.9773 ±0.0000 654 32.0
Kernelsfp160.04530.9773 ±0.0000 654 21.6

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-15; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6a5100ec, 2026-11-15; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15.

iBims-1

AbsRel / delta1

RundtypeAbsRel ↓δ1 ↑Δ AbsRel vs fp32 originalimagesms / image
Original fp32fp320.03740.9821 reference 100 109
Original fp16bf160.03740.9820 ±0.0000 100 48.7
Portable fp32fp320.03740.9821 ±0.0000 100 103
Portablefp160.03740.9821 ±0.0000 100 32.0
Kernelsfp160.03740.9821 ±0.0000 100 21.6

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-15; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6a5100ec, 2026-11-15; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15.

ETH3D two-view train left

AbsRel / delta1training data not stated: possibly seen

RundtypeAbsRel ↓δ1 ↑Δ AbsRel vs fp32 originalimagesms / image
Original fp32fp320.08960.9085 reference 27 108
Original fp16bf160.08960.9086 ±0.0000 27 49.3
Portable fp32not measured
Portablefp160.08960.9086 ±0.0000 27 31.4
Kernelsfp160.08960.9085 ±0.0000 27 25.0

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-15; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15.

Middlebury 2014 trainingH left

AbsRel / delta1training data not stated: possibly seen

RundtypeAbsRel ↓δ1 ↑Δ AbsRel vs fp32 originalimagesms / image
Original fp32fp320.03130.9948 reference 15 107
Original fp16bf160.03130.9948 ±0.0000 15 50.5
Portable fp32not measured
Portablefp160.03130.9948 ±0.0000 15 31.3
Kernelsfp160.03130.9948 ±0.0000 15 25.2

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-15; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15.

Closeness to the original

Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 1.5e-06 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.00029 and p99 <= 0.0012 (absolute error of the output points).

CheckToleranceMeasured
CPU fp32, max |diff|≤ 0.0000015 not recorded
GPU default dtypes, mean |diff|≤ 0.00029 0.00015
GPU default dtypes, p99 |diff|≤ 0.0012 0.00056

Recorded for pointdit-b-512 on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-14, with these kernels: dpt_fusedfastbb

Details

PointDiT (Xu et al., Google 2026): monocular geometry as generation; a pixel-space diffusion transformer denoises a point map in 3 Euler steps, conditioned on frozen DINOv3 features of the image. Outputs an affine-invariant point map and its depth.

Input / output
Input: one RGB image. Output: a depth or inverse-depth map.
What the numbers mean, and which models need the camera focal length

kind (a card field, result.kind) says what the numbers mean: metric = metres; scale_invariant, affine_invariant, affine_invariant_log (log depth) and affine_invariant_inverse are relative (unknown scale or shift, not metres). Metric models that use the focal length take model(image, intrinsics=K) (K [3, 3], pixels of the input): metric3d-vit-large and da3metric-large need it for correct metres (da3metric-large: metres = output x focal / 300; without intrinsics= a 300 px focal length is assumed, wrong for a real camera). unidepth-v2-vitl14, unik3d-vitl, depth-pro and MoGe estimate the camera themselves (given intrinsics= replace fx); zoedepth-nk and the relative models (da3mono-large too) take none. Normals-only models (kind none) return result.normals [B, H, W, 3] (unit vectors, camera frame: x right, y down, z forward).

Input size
any H x W >= 1, resized or padded inside the model; tested up to 3000x4000 (843 MiB allocated)
Options
  • tokens=1024>= 1· changes the output
  • steps=3>= 1· changes the output
Outputs
points, depth
Checkpoint
arch="b", encoder="dinov3-vitb16"
CUDA kernels
  • fastbbfast backbone kernels: flash attention, window attention, instance norm and rotary embeddings for ViT and similar backbones
  • dpt_fusedfused fp16 matrix multiplies and convolutions for DINOv2 backbones and DPT depth heads
Apple (MPS)
fp32, fp16

Limitations

  • Affine-invariant output: points up to an unknown scale and 3-D shift, depth (their z) up to scale and shift; align before use.
  • Trained at 512 x 512: inputs are resized to about `tokens` 16-pixel patches at their aspect ratio (bilinear) and the point map is resized back with nearest, as the original's wild-image path.
  • Kernels mode captures CUDA graphs: the graph's private memory pool and static outputs stay reserved after the call, so peak reserved is above portable by ~85 MiB (b), more for l (peak allocated is equal or lower); `cuda_graphs=false` or `release_graphs()` frees it at the cost of launch overhead.
Full reference, generated from the repository

Google's PointDiT: a diffusion transformer that denoises a 3D point map in three steps from one image, conditioned on frozen DINOv3 features. Code and weights are Apache-2.0; the DINOv3 encoder it loads is under the DINOv3 License.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
pointdit-b-512Middlebury 2014 half (994x1482)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
105
48.7
31.1
20.5
23.7
15.5
977 / 1006
1214 / 1246
446 / 504
462 / 630
465 / 524
795 / 898
pointdit-l-512Middlebury 2014 half (994x1482)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
335
112
94.6
61.3
76.0
49.1
3092 / 3148
3953 / 4010
1297 / 1430
1310 / 1676
1316 / 1430
2474 / 2626
Details
  • Original code: https://github.com/google-research/pointdit at 11f53a30aeb4
  • Code licence: Apache-2.0 (permissive); files: LICENSE
  • Weights licence: LicenseRef-DINOv3-License
  • Licence evidence: https://github.com/google-research/pointdit/blob/11f53a30aeb4d34d49d2beda655ee6f19b2ad7f8/LICENSE (Apache-2.0); https://huggingface.co/haofeixu/pointdit (license: apache-2.0); encoder: https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m (DINOv3 License, gated)
  • Training data: Mixture of synthetic and real point-map datasets ("mixdata") per the PointDiT paper; the DINOv3 encoder: LVD-1689M (not released)
  • Licence note: The PointDiT code and weights are Apache-2.0; the frozen DINOv3 ViT-B/16 and ViT-L/16 encoders are under the DINOv3 License (Meta: attribution and conditions apply; gated on Hugging Face); read it before commercial use.
  • Options: tokens = 1024 (>= 1), steps = 3 (>= 1)
  • Kernel libraries: fastbb, dpt_fused
  • Variants: pointdit-b-512, pointdit-l-512
  • Outputs: points, depth; kind: affine_invariant
  • Input size: any H x W >= 1, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): pointdit-b-512 3000x4000 (0.8 GiB peak allocated, 1.0 reserved); pointdit-l-512 3000x4000 (1.6 GiB peak allocated, 2.2 reserved)
  • pointdit-b-512 994x1482, first call: portable + torch.compile 34 s (cold Inductor cache), TensorRT fp16 86 s (ONNX + engine).
  • pointdit-l-512 994x1482, first call: portable + torch.compile 57 s (cold Inductor cache), TensorRT fp16 150 s (ONNX + engine).
  • pointdit-b-512 original (fp32) 994x1482: upstream wild-image resize (~1024 tokens), 3 Euler steps
  • pointdit-b-512 original 994x1482: the authors' mixed-precision option (bfloat16); upstream wild-image resize (~1024 tokens), 3 Euler steps; authors bf16 autocast
  • pointdit-l-512 original (fp32) 994x1482: upstream wild-image resize (~1024 tokens), 3 Euler steps
  • pointdit-l-512 original 994x1482: the authors' mixed-precision option (bfloat16); upstream wild-image resize (~1024 tokens), 3 Euler steps; authors bf16 autocast
  • pointdit-b-512 load / first call: original 2.57 s / 0.111 s, portable 1.67 s / 0.184 s (518x770), kernels 1.74 s / 0.155 s (518x770)
  • pointdit-l-512 load / first call: original 9.47 s / 0.119 s, portable 3.96 s / 0.247 s (518x770), kernels 3.91 s / 0.234 s (518x770)
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5

Measured

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portableportable fp32kernelsnote
pointdit-b-512NYUv2 test, AbsRel / delta10.0453 / 0.97730.0454 / 0.97730.0453 / 0.97730.0453 / 0.97730.0453 / 0.9773
pointdit-b-512iBims-1, AbsRel / delta10.0374 / 0.98210.0374 / 0.98200.0374 / 0.98210.0374 / 0.98210.0374 / 0.9821
pointdit-b-512ETH3D two-view train left, AbsRel / delta10.0896 / 0.90850.0896 / 0.90860.0896 / 0.90860.0896 / 0.9085possibly in the training data
pointdit-b-512Middlebury 2014 trainingH left, AbsRel / delta10.0313 / 0.99480.0313 / 0.99480.0313 / 0.99480.0313 / 0.9948possibly in the training data
pointdit-l-512NYUv2 test, AbsRel / delta10.0410 / 0.98000.0411 / 0.98000.0410 / 0.98000.0410 / 0.98000.0410 / 0.9800
pointdit-l-512iBims-1, AbsRel / delta10.0313 / 0.98480.0313 / 0.98470.0313 / 0.98480.0313 / 0.98480.0313 / 0.9848
pointdit-l-512ETH3D two-view train left, AbsRel / delta10.0748 / 0.92640.0748 / 0.92780.0748 / 0.92650.0748 / 0.9261possibly in the training data
pointdit-l-512Middlebury 2014 trainingH left, AbsRel / delta10.0230 / 0.99520.0230 / 0.99510.0230 / 0.99520.0230 / 0.9952possibly in the training data

Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6a5100ec, 2026-11-15; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2fb984b3, 2026-11-15.

Parity

CPU, float32: max |diff| <= 1.5e-06 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.00029 and p99 <= 0.0012 (absolute error of the output points).

variantkernels error vs original, mean / p99GPU, torchkernels used
pointdit-b-5120.00015 / 0.00056RTX 5060 Ti, torch 2.14.1+cu130dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120
pointdit-l-5120.00012 / 0.00045RTX 5060 Ti, torch 2.14.1+cu130dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120

Limitations

  • Affine-invariant output: points up to an unknown scale and 3-D shift, depth (their z) up to scale and shift; align before use.
  • Trained at 512 x 512: inputs are resized to about tokens 16-pixel patches at their aspect ratio (bilinear) and the point map is resized back with nearest, as the original's wild-image path.
  • Kernels mode captures CUDA graphs: the graph's private memory pool and static outputs stay reserved after the call, so peak reserved is above portable by ~85 MiB (b), more for l (peak allocated is equal or lower); cuda_graphs=false or release_graphs() frees it at the cost of launch overhead.

Example output

No example rendered for this checkpoint yet.