DINOv2
Meta's DINOv2: self-supervised image features (patch and class tokens) in four sizes, with and without register tokens.
stable Fork of facebookresearch/dinov2@7764ea0 · backends: portable, kernels
- Paper DINOv2: Learning Robust Visual Features without Supervision TMLR 2024 · first released 14 Apr 2023
Paper 2023 Added in 0.1.0 Evaluated with 0.2.0
Use it
pip install fastforks Lean install with only what this model needs: pip install fastforks-core
from fastforks import features
model = features.load("dinov2-vits14") # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.last_hidden_state # also: pooler_output, hidden_states
result.backend # what ran: mode, device, dtype, kernels fastforks bench dinov2-vits14 Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.
Speed and memory
Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.
Input: synthetic, 224x224 (the task’s standard size)
| Backend | dtype | Time per call (ms) | vs original fp32 | Peak memory allocated / reserved | Load (s) | First call (s) |
|---|---|---|---|---|---|---|
| Original fp32 | fp32 | 3.30 | – | 121 MiB/132 MiB | 0.3 | 0.11 |
| Original fp16torch.autocast(fp16) around the forward | fp16 | 4.40 | 0.75× | 160 MiB/166 MiB | 0.4 | 0.17 |
| Portable fp32 | fp32 | not measured | – | – | – | – |
| Portable | fp16 | 2.95 | 1.1× | 79 MiB/98 MiB | 0.1 | 0.12 |
| Kernels | fp16 | 0.827 | 4.0× | 80 MiB/122 MiB | 1.0 | 0.10 |
| Portable + torch.compile | 2.02 | 1.6× | 111 MiB/140 MiB | 10 (build) | – | |
| TensorRT fp16 | fp16 | 0.601 | 5.5× | 64 MiBenginenot comparable | 23 (build) | – |
Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-01 to 2026-11-12. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two
times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.
Other devices
| Device | Backend | dtype | ms / call | Peak memory | Output vs reference | Input |
|---|---|---|---|---|---|---|
| Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-12 | Original | fp32 | 12.8 | 476 MiB process | matches | 224x224 |
| Portable | fp16 | 11.2 | 341 MiB process | matches | 224x224 | |
| Portable | fp32 | 12.2 | 381 MiB process | matches | 224x224 | |
| Kernels | fp16 | not implemented | – | 224x224 | ||
| NVIDIA L4 CUDA, torch 2.14.1+cu130, 2026-11-07 | Portable | fp16 | 4.34 | 54 MiB/58 MiB | – | 224x224 |
| Kernels | fp16 | 1.05 | 63 MiB/82 MiB | – | 224x224 | |
| NVIDIA L40S CUDA, torch 2.14.1+cu130, 2026-11-07 | Portable | fp16 | 3.39 | 54 MiB/58 MiB | – | 224x224 |
| Kernels | fp16 | 0.721 | 63 MiB/82 MiB | – | 224x224 | |
| NVIDIA A10 CUDA, torch 2.14.1+cu130, 2026-11-07 | Portable | fp16 | 2.96 | 54 MiB/58 MiB | – | 224x224 |
| Kernels | fp16 | 0.996 | 63 MiB/82 MiB | – | 224x224 | |
| NVIDIA A100-SXM4-40GB CUDA, torch 2.14.1+cu130, 2026-11-07 | Portable | fp16 | 3.33 | 55 MiB/58 MiB | – | 224x224 |
| Kernels | fp16 | 1.40 | 63 MiB/102 MiB | – | 224x224 | |
| NVIDIA RTX PRO 6000 CUDA, torch 2.14.1+cu130, 2026-11-08 | Portable | fp16 | 1.07 | 78 MiB/98 MiB | – | 224x224 |
| Kernels | fp16 | 0.645 | 111 MiB/142 MiB | – | 224x224 | |
CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.
Licence and weights
Commercial use allowed
- Code
- Apache-2.0 LICENSE
- Weights
- Apache-2.0 source
- Training data
- LVD-142M: 142M images retrieved from web crawls by similarity to curated datasets (ImageNet-22k/1k train, Google Landmarks, fine-grained sets), self-supervised (no labels), per the paper; not released
Our reading of the licences, not legal advice.
Weights
dinov2_vits14_pretrain.pth 88.3 MB· hosted on Meta (fbaipublicfiles)fastforks downloads and checks these files on first use (fastforks download dinov2-vits14).
Accuracy
The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.
ImageNet-1k val
kNN top-1 % (k = 20, 10000 labelled / 5000 test images)
| Run | dtype | top-1 % ↑ | Δ top-1 % vs fp32 original | images | ms / image |
|---|---|---|---|---|---|
| Original fp32 | fp32 | 59.20 | reference | 5000 | 1.51 |
| Original fp16 | fp16 | 59.22 | +0.02 | 5000 | 0.566 |
| Portable fp32 | fp32 | 59.22 | +0.02 | 5000 | 1.54 |
| Portable | fp16 | 59.26 | +0.06 | 5000 | 0.468 |
| Kernels | fp16 | 59.16 | −0.04 | 5000 | 0.347 |
Two disjoint seeded subsets of the val folders; each image shorter side 256 (bicubic), centre crop 224, the CLS / pooled feature; cosine kNN, vote weighted by similarity. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-06; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 7c91993, 2026-10-31; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.
Closeness to the original
Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 2.5 % and p99 <= 12 % of the reference's median |value| (absolute error of the output last_hidden_state).
| Check | Tolerance | Measured |
|---|---|---|
| CPU fp32, max |diff| | ≤ 0.0001 (relative) | not recorded |
| cpu_rtol | ≤ 0.00002 (relative) | not recorded |
| GPU default dtypes, mean |diff| | ≤ 0.025 (relative) | 0.01036 (relative) |
| GPU default dtypes, p99 |diff| | ≤ 0.12 (relative) | 0.03758 (relative) |
Recorded for dinov2-vits14 on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-12, with these kernels: dpt_fusedfastbb
Details
DINOv2 (Oquab et al. 2023, Meta): self-supervised ViT backbones; patch + cls token features at any size; -reg = with 4 register tokens.
- Input / output
- Input: one RGB image. Output: dense patch features and a pooled image feature.
- Input size
- any H x W >= 1, resized or padded inside the model; tested up to 3000x4000 (846 MiB allocated)
- Options
fp16_accumulate=false
- Outputs
last_hidden_state,pooler_output,hidden_states- Checkpoint
arch="vits",registers=0- CUDA kernels
fastbbfast backbone kernels: flash attention, window attention, instance norm and rotary embeddings for ViT and similar backbonesdpt_fusedfused fp16 matrix multiplies and convolutions for DINOv2 backbones and DPT depth heads
- Apple (MPS)
- fp32, fp16
Full reference, generated from the repository
Meta's DINOv2: self-supervised image features (patch and class tokens) in four sizes, with and without register tokens.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
dinov2-vits14 | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 3.30 4.40 2.95 0.827 2.02 0.601 | 121 / 132 160 / 166 79 / 98 80 / 122 111 / 140 141 / 160 |
dinov2-vits14-reg | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 4.11 5.59 2.98 0.832 2.19 0.614 | 121 / 132 160 / 168 80 / 98 80 / 102 111 / 118 145 / 164 |
dinov2-vitb14 | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 6.90 4.50 2.92 1.50 2.20 1.30 | 371 / 386 530 / 548 205 / 226 208 / 228 240 / 266 367 / 384 |
dinov2-vitb14-reg | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 7.00 4.30 2.94 1.52 2.30 1.33 | 371 / 386 530 / 548 207 / 226 208 / 228 240 / 246 399 / 402 |
dinov2-vitl14 | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 22.8 9.80 6.13 4.62 6.03 4.25 | 1205 / 1224 1777 / 1784 622 / 644 627 / 648 661 / 666 1235 / 1262 |
dinov2-vitl14-reg | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 22.5 10.7 6.38 4.60 6.17 4.27 | 1205 / 1226 1777 / 1786 624 / 646 627 / 648 661 / 666 1237 / 1242 |
dinov2-vitg14 | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 86.4 33.9 18.9 12.8 18.3 12.0 | 4389 / 4406 6542 / 6548 2216 / 2230 2222 / 2250 2267 / 2290 4392 / 4410 |
dinov2-vitg14-reg | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 86.6 34.2 18.7 12.7 18.3 12.0 | 4389 / 4406 6542 / 6568 2217 / 2228 2222 / 2250 2267 / 2290 4400 / 4418 |
Details
- Original code: https://github.com/facebookresearch/dinov2 at
7764ea0 - Code licence: Apache-2.0 (permissive); files:
LICENSE - Weights licence: Apache-2.0
- Licence evidence: https://github.com/facebookresearch/dinov2/blob/7764ea0/README.md (License section: code and model weights under Apache 2.0)
- Training data: LVD-142M: 142M images retrieved from web crawls by similarity to curated datasets (ImageNet-22k/1k train, Google Landmarks, fine-grained sets), self-supervised (no labels), per the paper; not released
- Options:
fp16_accumulate= False (precision only) - Kernel libraries:
fastbb,dpt_fused - Variants:
dinov2-vits14,dinov2-vits14-reg,dinov2-vitb14,dinov2-vitb14-reg,dinov2-vitl14,dinov2-vitl14-reg,dinov2-vitg14,dinov2-vitg14-reg - Outputs: last_hidden_state, pooler_output, hidden_states
- Input size: any H x W >= 1, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
dinov2-vits143000x4000 (0.8 GiB peak allocated, 1.2 reserved);dinov2-vits14-reg3000x4000 (0.8 GiB peak allocated, 1.2 reserved);dinov2-vitb143000x4000 (1.4 GiB peak allocated, 2.2 reserved);dinov2-vitb14-reg3000x4000 (1.4 GiB peak allocated, 2.2 reserved);dinov2-vitl143000x4000 (2.2 GiB peak allocated, 3.1 reserved);dinov2-vitl14-reg3000x4000 (2.2 GiB peak allocated, 3.1 reserved);dinov2-vitg143000x4000 (4.8 GiB peak allocated, 5.9 reserved);dinov2-vitg14-reg3000x4000 (4.8 GiB peak allocated, 5.9 reserved) dinov2-vits14224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 23 s (ONNX + engine).dinov2-vits14-reg224x224, first call: portable + torch.compile 6 s (cold Inductor cache), TensorRT fp16 32 s (ONNX + engine).dinov2-vitb14224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 25 s (ONNX + engine).dinov2-vitb14-reg224x224, first call: portable + torch.compile 6 s (cold Inductor cache), TensorRT fp16 29 s (ONNX + engine).dinov2-vitl14224x224, first call: portable + torch.compile 13 s (cold Inductor cache), TensorRT fp16 34 s (ONNX + engine).dinov2-vitl14-reg224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 42 s (ONNX + engine).dinov2-vitg14224x224, first call: portable + torch.compile 19 s (cold Inductor cache), TensorRT fp16 46 s (ONNX + engine).dinov2-vitg14-reg224x224, first call: portable + torch.compile 15 s (cold Inductor cache), TensorRT fp16 57 s (ONNX + engine).dinov2-vits14original 224x224: torch.autocast(fp16) around the forwarddinov2-vits14-regoriginal (fp32) 224x224, original 224x224: dinov2 forward_features CLSdinov2-vitb14original 224x224: torch.autocast(fp16) around the forwarddinov2-vitb14-regoriginal 224x224: torch.autocast(fp16) around the forwarddinov2-vitl14original 224x224: torch.autocast(fp16) around the forwarddinov2-vitl14-regoriginal (fp32) 224x224, original 224x224: dinov2 forward_features CLSdinov2-vitg14original 224x224: torch.autocast(fp16) around the forwarddinov2-vitg14-regoriginal (fp32) 224x224, original 224x224: dinov2 forward_features CLSdinov2-vits14load / first call: original 0.41 s / 0.174 s, portable 0.07 s / 0.118 s, kernels 0.07 s / 0.153 sdinov2-vits14-regload / first call: original 1.32 s / 0.107 s, portable 0.21 s / 0.126 s, kernels 0.08 s / 0.167 sdinov2-vitb14load / first call: original 1.51 s / 0.006 s, portable 0.27 s / 0.205 s, kernels 0.19 s / 0.191 sdinov2-vitb14-regload / first call: original 1.71 s / 0.035 s, portable 0.59 s / 0.042 s, kernels 0.21 s / 0.205 sdinov2-vitl14load / first call: original 7.06 s / 0.013 s, portable 2.3 s / 0.094 s, kernels 2.31 s / 0.123 sdinov2-vitl14-regload / first call: original 4.6 s / 0.068 s, portable 2.08 s / 0.093 s, kernels 2.11 s / 0.123 sdinov2-vitg14load / first call: original 19.29 s / 0.046 s, portable 4.37 s / 0.104 s, kernels 2.22 s / 0.162 sdinov2-vitg14-regload / first call: original 14.67 s / 0.089 s, portable 4.32 s / 0.033 s, kernels 15.39 s / 0.149 s- Protocol,
dinov2-vits14,dinov2-vits14-reg,dinov2-vitb14,dinov2-vitb14-reg,dinov2-vitl14,dinov2-vitl14-reg,dinov2-vitg14: first call, 3 warm-up, peak reset, median of 3 rounds x median of 5 - Protocol,
dinov2-vitg14-reg: median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
Two disjoint seeded subsets of the val folders; each image shorter side 256 (bicubic), centre crop 224, the CLS / pooled feature; cosine kNN, vote weighted by similarity. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | portable fp32 | kernels | note |
|---|---|---|---|---|---|---|---|
dinov2-vits14 | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 59.20 | 59.22 | 59.26 | 59.22 | 59.16 | |
dinov2-vits14-reg | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 59.98 | 60.00 | 59.96 | 59.98 | 59.96 | |
dinov2-vitb14 | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 67.80 | 67.76 | 67.74 | 67.80 | 67.78 | |
dinov2-vitb14-reg | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 69.14 | 69.18 | 69.14 | 69.14 | 69.18 | |
dinov2-vitl14 | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 72.12 | 72.16 | 72.18 | 72.12 | 72.14 | |
dinov2-vitl14-reg | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 73.08 | 73.08 | 73.06 | 73.08 | 73.08 | |
dinov2-vitg14 | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 70.64 | 70.66 | 70.64 | 70.64 | 70.68 | |
dinov2-vitg14-reg | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 72.60 | 72.58 | 72.60 | 72.60 | 72.64 |
Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 7c91993, 2026-10-31; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 2.5 % and p99 <= 12 % of the reference's median |value| (absolute error of the output last_hidden_state).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
dinov2-vits14 | 0.01036 / 0.03758 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vits14-reg | 0.00541 / 0.0198 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitb14 | 0.01295 / 0.05042 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitb14-reg | 0.01032 / 0.03738 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitl14 | 0.01558 / 0.08667 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitl14-reg | 0.01356 / 0.05036 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitg14 | 0.00785 / 0.0449 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
dinov2-vitg14-reg | 0.00779 / 0.04094 | RTX 5060 Ti, torch 2.14.1+cu130 | dpt_fused@48e0065bb7cf:sm_120, fastbb@7d058322634d:sm_120 |
Example output
One fixed input through this checkpoint, the same for every model of the task (kernels · float16 · NVIDIA GeForce RTX 5060 Ti).
Input: Chelsea the cat, Stéfan van der Walt (CC0).