RF-DETR (N / S / M / L detection, Seg N to 2XL)
Roboflow RF-DETR (N, S, M, L detection; Seg N to 2XL with instance masks; COCO): real-time end-to-end detector with a windowed DINOv2 backbone and deformable decoder, no NMS. Apache-2.0 code and weights.
experimental Fork of roboflow/rf-detr@5b22dfc · backends: portable, kernels
- Paper RF-DETR: Neural Architecture Search for Real-Time Detection Transformers ICLR 2026 · first released 12 Nov 2025
Paper 2025 Added in 0.2.0 Evaluated with 0.2.0
Use it
pip install fastforks Lean install with only what this model needs: pip install "fastforks-core[torchvision,transformers]"
from fastforks import detection
model = detection.load("rf-detr-n-coco") # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.boxes # also: scores, labels, masks
result.backend # what ran: mode, device, dtype, kernels fastforks bench rf-detr-n-coco Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.
Speed and memory
Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.
Input: synthetic, 800x1344 (the task’s standard size)
| Backend | dtype | Time per call (ms) | vs original fp16 | Peak memory allocated / reserved | Load (s) | First call (s) |
|---|---|---|---|---|---|---|
| Original fp32 | fp32 | not measured | – | – | – | – |
| Original fp16the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True) | fp16 | 10.5 | – | 118 MiB/160 MiB | 2.7 | 0.07 |
| Portable fp32 | fp32 | not measured | – | – | – | – |
| Portable | fp16 | 9.39 | 1.1× | 117 MiB/140 MiB | 2.6 | 0.34 |
| Kernels | fp16 | 2.39 | 4.4× | 117 MiB/142 MiB | 3.0 | 0.37 |
| Portable + torch.compile | 4.58 | 2.3× | 141 MiB/156 MiB | 26 (build) | – | |
| TensorRT fp16 | fp16 | 1.15 | 9.1× | 74 MiBenginenot comparable | 60 (build) | – |
Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-12 to 2026-11-16. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two
times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.
Other devices
| Device | Backend | dtype | ms / call | Peak memory | Output vs reference | Input |
|---|---|---|---|---|---|---|
| Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-13 | Original | fp32 | 39.6 | 2,117 MiB process | – | 800x1344 |
| Portable | fp16 | 30.3 | 1,630 MiB process | matches | 800x1344 | |
| Portable | fp32 | 31.8 | 1,718 MiB process | matches | 800x1344 | |
| Kernels | fp16 | not implemented | – | 800x1344 | ||
CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.
Licence and weights
Commercial use allowed
- Code
- Apache-2.0 LICENSE
- Weights
- Apache-2.0 source
- Training data
-
- COCO 2017 (train, instance annotations) CC-BY-4.0
- Objects365 (train, box annotations) CC-BY-4.0, non-commercial
- LVD-142M (DINOv2's curated web-image set, not released) not stated
Our reading of the licences, not legal advice.
Weights
rf-detr-nano.pth 366 MB· hosted on repo.roboflow.comfastforks downloads and checks these files on first use (fastforks download rf-detr-n-coco).
Accuracy
The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.
COCO val2017
every 10th image, box mAP
| Run | dtype | box mAP ↑ | Δ box mAP vs fp32 original | images | ms / image |
|---|---|---|---|---|---|
| Original fp32 | fp32 | 48.18 | reference | 500 | 10.9 |
| Original fp16 | not measured | ||||
| Portable fp32 | fp32 | 48.19 | +0.01 | 500 | 11.6 |
| Portable | not measured | ||||
| Kernels | not measured | ||||
pycocotools COCOeval of the top 100 detections per image. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-18; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 30c7a8e9, 2026-11-18.
Closeness to the original
Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.1 and p99 <= 4.2 (absolute error of the output boxes).
| Check | Tolerance | Measured |
|---|---|---|
| CPU fp32, max |diff| | ≤ 0.0001 | not recorded |
| GPU default dtypes, mean |diff| | ≤ 1.1 | 0.04829 |
| GPU default dtypes, p99 |diff| | ≤ 4.2 | 0.1498 |
Recorded for rf-detr-n-coco on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-13, with these kernels: fastbbfvm_conv
Details
RF-DETR (Roboflow 2025): real-time end-to-end detection with a DINOv2-small backbone using windowed attention, a single-scale projector and a deformable-attention decoder over 100 to 300 queries, no NMS; boxes, scores and COCO classes at the input size, plus instance masks for the Seg variants. The image is resized to a square (384 to 768 px per variant). The portable `module()` is the whole network (normalised boxes, class logits and, for Seg, mask logits).
- Input / output
- Input: one RGB image. Output: boxes, class labels, scores and instance masks.
- Input size
- any H x W >= 32, padded to a multiple of 32; tested up to 3000x4000 (505 MiB allocated)
- Options
score_threshold=0.50.0 to 1.0· changes the outputmax_detections=100>= 1· changes the output
- Outputs
boxes,scores,labels,masks- Checkpoint
dec_layers=2,resolution=384,positional_encoding_size=24- CUDA kernels
fastbbfast backbone kernels: flash attention, window attention, instance norm and rotary embeddings for ViT and similar backbonesfvm_convfused convolutions for image classifiers
- Apple (MPS)
- fp32, fp16
Limitations
- Requires the optional transformers and torchvision libraries.
- Only the Apache-2.0 checkpoints are included; the PML-1.0 detection variants (A / F / P / XL / 2XL) are not.
- torch.compile uses upstream positional-interpolation graph breaks; fullgraph=True is unsupported.
- Kernels keep CUDA graphs: their private pool and static outputs stay reserved, so peak reserved is ~130 MiB above portable (peak allocated is not); `cuda_graphs=false` or `release_graphs()` frees it.
Full reference, generated from the repository
Roboflow RF-DETR (N, S, M, L detection; Seg N to 2XL with instance masks; COCO): real-time end-to-end detector with a windowed DINOv2 backbone and deformable decoder, no NMS. Apache-2.0 code and weights.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
rf-detr-n-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 10.5 9.39 2.39 4.58 1.15 | 118 / 160 117 / 140 117 / 142 141 / 156 179 / 208 |
rf-detr-s-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 11.2 10.0 3.33 4.56 1.83 | 189 / 238 128 / 148 128 / 172 150 / 162 188 / 214 |
rf-detr-m-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 11.8 10.6 3.86 4.92 2.10 | 140 / 174 136 / 156 136 / 178 156 / 166 194 / 216 |
rf-detr-l-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 11.8 10.6 5.30 5.32 2.87 | 219 / 240 149 / 184 149 / 206 164 / 182 203 / 222 |
rf-detr-seg-n-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 13.0 11.7 3.68 6.59 1.52 | 255 / 298 125 / 148 125 / 170 158 / 166 189 / 218 |
rf-detr-seg-s-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 13.0 12.0 4.28 6.55 2.00 | 327 / 382 132 / 154 132 / 178 162 / 190 197 / 224 |
rf-detr-seg-m-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 14.4 13.3 5.31 6.86 2.50 | 381 / 424 141 / 180 141 / 202 172 / 194 208 / 228 |
rf-detr-seg-l-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 15.3 14.1 6.60 7.08 3.30 | 459 / 492 150 / 168 150 / 208 190 / 208 218 / 234 |
rf-detr-seg-xl-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 18.7 18.1 10.4 10.8 5.20 | 182 / 248 177 / 206 178 / 268 212 / 246 302 / 326 |
rf-detr-seg-2xl-coco | synthetic (800x1344) | original portable kernels portable + torch.compile TensorRT fp16 | 25.6 26.0 16.0 16.9 8.62 | 287 / 352 207 / 228 209 / 312 245 / 308 334 / 366 |
Details
- Original code: https://github.com/roboflow/rf-detr at
5b22dfc09406 - Code licence: Apache-2.0 (permissive); files:
LICENSE - Weights licence: Apache-2.0
- Licence evidence: https://github.com/roboflow/rf-detr/blob/5b22dfc0940697ce1abdfa681a5f7392b3eddbfb/README.md#license
- Training data: COCO 2017 (train, instance annotations), Objects365 (train, box annotations), LVD-142M (DINOv2's curated web-image set, not released) (COCO 2017 train (README); the RF-DETR paper (arXiv:2511.09554) pretrains detection on Objects365; the DINOv2-small backbone originates from LVD-142M pretraining. Full intermediate training provenance is not stated.)
- Options:
score_threshold= 0.5 (0.0 to 1.0),max_detections= 100 (>= 1) - Kernel libraries:
fastbb,fvm_conv - Variants:
rf-detr-n-coco,rf-detr-s-coco,rf-detr-m-coco,rf-detr-l-coco,rf-detr-seg-n-coco,rf-detr-seg-s-coco,rf-detr-seg-m-coco,rf-detr-seg-l-coco,rf-detr-seg-xl-coco,rf-detr-seg-2xl-coco - Outputs: boxes, scores, labels, masks; label_set:
coco-80 - Input size: any H x W >= 32, padded to a multiple of 32
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
rf-detr-n-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-s-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-m-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-l-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-n-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-s-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-m-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-l-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-xl-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved);rf-detr-seg-2xl-coco3000x4000 (0.5 GiB peak allocated, 0.5 reserved) rf-detr-n-coco800x1344, first call: portable + torch.compile 26 s (cold Inductor cache), TensorRT fp16 60 s (ONNX + engine).rf-detr-s-coco800x1344, first call: portable + torch.compile 24 s (cold Inductor cache), TensorRT fp16 52 s (ONNX + engine).rf-detr-m-coco800x1344, first call: portable + torch.compile 28 s (cold Inductor cache), TensorRT fp16 55 s (ONNX + engine).rf-detr-l-coco800x1344, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 66 s (ONNX + engine).rf-detr-seg-n-coco800x1344, first call: portable + torch.compile 31 s (cold Inductor cache), TensorRT fp16 54 s (ONNX + engine).rf-detr-seg-s-coco800x1344, first call: portable + torch.compile 30 s (cold Inductor cache), TensorRT fp16 64 s (ONNX + engine).rf-detr-seg-m-coco800x1344, first call: portable + torch.compile 32 s (cold Inductor cache), TensorRT fp16 72 s (ONNX + engine).rf-detr-seg-l-coco800x1344, first call: portable + torch.compile 32 s (cold Inductor cache), TensorRT fp16 68 s (ONNX + engine).rf-detr-seg-xl-coco800x1344, first call: portable + torch.compile 35 s (cold Inductor cache), TensorRT fp16 92 s (ONNX + engine).rf-detr-seg-2xl-coco800x1344, first call: portable + torch.compile 45 s (cold Inductor cache), TensorRT fp16 69 s (ONNX + engine).rf-detr-n-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-s-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-m-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-l-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-n-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-s-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-m-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-l-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-xl-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-seg-2xl-cocooriginal 800x1344: the authors' float16 model option; pinned RF-DETR torchvision resize/normalize, original network and PostProcess; authors inference(compile=False, dtype=float16, inplace=True)rf-detr-n-cocoload / first call: original 5.93 s / 0.557 s, portable 2.59 s / 0.351 s, kernels 2.67 s / 0.346 srf-detr-s-cocoload / first call: original 2.74 s / 0.016 s, portable 2.61 s / 0.349 s, kernels 2.65 s / 0.345 srf-detr-m-cocoload / first call: original 2.77 s / 0.016 s, portable 3.59 s / 0.425 s, kernels 2.96 s / 0.346 srf-detr-l-cocoload / first call: original 1.1 s / 0.019 s, portable 6.42 s / 0.466 s, kernels 3.36 s / 0.466 srf-detr-seg-n-cocoload / first call: original 0.43 s / 0.026 s, portable 2.92 s / 0.36 s, kernels 2.84 s / 0.344 srf-detr-seg-s-cocoload / first call: original 0.42 s / 0.017 s, portable 3.18 s / 0.351 s, kernels 2.79 s / 0.337 srf-detr-seg-m-cocoload / first call: original 0.44 s / 0.018 s, portable 3.95 s / 0.399 s, kernels 2.74 s / 0.344 srf-detr-seg-l-cocoload / first call: original 0.44 s / 0.021 s, portable 3.12 s / 0.336 s, kernels 2.88 s / 0.328 srf-detr-seg-xl-cocoload / first call: original 0.61 s / 0.036 s, portable 3.2 s / 0.333 s, kernels 2.92 s / 0.327 srf-detr-seg-2xl-cocoload / first call: original 0.47 s / 0.06 s, portable 3.37 s / 0.335 s, kernels 5.77 s / 0.358 s- Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5
Measured
pycocotools COCOeval of the top 100 detections per image. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | portable fp32 | kernels | note |
|---|---|---|---|---|---|---|---|
rf-detr-n-coco | COCO val2017, every 10th image, box mAP | 48.18 | 48.19 | ||||
rf-detr-m-coco | COCO val2017, every 10th image, box mAP | 54.48 | 54.42 | 54.42 | 54.42 | 54.40 | |
rf-detr-m-coco | COCO val2017, all 5000 images, box mAP | 54.53 | 54.51 | 54.51 | |||
rf-detr-seg-m-coco | COCO val2017, every 10th image, box mAP / mask mAP | 53.21 / 44.24 | 53.30 / 44.30 | 53.30 / 44.30 | 53.35 / 44.32 | 53.49 / 44.38 |
Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 7cecd6cb, 2026-11-12; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 30c7a8e9 / 44640884, 2026-11-17 to 2026-11-18; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 7cecd6cb, 2026-11-12.
Parity
CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.1 and p99 <= 4.2 (absolute error of the output boxes).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
rf-detr-n-coco | 0.04829 / 0.1498 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-s-coco | 0.06718 / 0.1126 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-m-coco | 0.09944 / 0.22606 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-l-coco | 0.15787 / 0.32693 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-n-coco | 0.25982 / 1.06715 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-s-coco | 0.12115 / 0.31712 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-m-coco | 0.13842 / 0.39567 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-l-coco | 0.26359 / 0.91166 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-xl-coco | 0.73043 / 2.97183 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
rf-detr-seg-2xl-coco | 0.37543 / 1.74394 | RTX 5060 Ti, torch 2.14.1+cu130 | fastbb@7d058322634d:sm_120, fvm_conv@f1d064e539e1:sm_120 |
Limitations
- Requires the optional transformers and torchvision libraries.
- Only the Apache-2.0 checkpoints are included; the PML-1.0 detection variants (A / F / P / XL / 2XL) are not.
- torch.compile uses upstream positional-interpolation graph breaks; fullgraph=True is unsupported.
- Kernels keep CUDA graphs: their private pool and static outputs stay reserved, so peak reserved is ~130 MiB above portable (peak allocated is not);
cuda_graphs=falseorrelease_graphs()frees it.
Example output
No example rendered for this checkpoint yet.