ConvNeXt
Meta's ConvNeXt (tiny, small, base, large) trained on ImageNet-22k, without the classifier: the last stage's feature map as tokens and a pooled image embedding. Apache-2.0 weights.
experimental Fork of huggingface/pytorch-image-models@0df212b · backends: portable, kernels
- Paper A ConvNet for the 2020s CVPR 2022 · first released 10 Jan 2022
- Authors’ code facebookresearch/ConvNeXt (PyTorch), not the code forked here: timm carries the architecture and the converted authors' checkpoints in its own, simpler API.
Paper 2022 Added in 0.2.0 Evaluated with 0.2.0
Use it
pip install fastforks Lean install with only what this model needs: pip install "fastforks-core[timm]"
from fastforks import features
model = features.load("convnext_tiny.fb_in22k") # kernels on a supported GPU, else portable PyTorch
result = model(image)
result.last_hidden_state # also: pooler_output
result.backend # what ran: mode, device, dtype, kernels fastforks bench convnext_tiny.fb_in22k Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.
Speed and memory
Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.
Input: synthetic, 224x224 (the task’s standard size)
| Backend | dtype | Time per call (ms) | vs original fp32 | Peak memory allocated / reserved | Load (s) | First call (s) |
|---|---|---|---|---|---|---|
| Original fp32timm num_classes=0 (pooled, head norm) | fp32 | 3.89 | – | 151 MiB/170 MiB | 1.8 | 0.76 |
| Original fp16timm num_classes=0 (pooled, head norm) | fp16 | 5.91 | 0.66× | 194 MiB/214 MiB | 0.4 | 0.28 |
| Portable fp32 | fp32 | not measured | – | – | – | – |
| Portable | fp16 | 3.26 | 1.2× | 93 MiB/108 MiB | 1.6 | 0.25 |
| Kernels | fp16 | 1.10 | 3.5× | 88 MiB/110 MiB | 1.7 | 0.22 |
| Portable + torch.compile | 2.08 | 1.9× | 148 MiB/170 MiB | 12 (build) | – | |
| TensorRT fp16 | fp16 | 0.726 | 5.4× | 72 MiBenginenot comparable | 34 (build) | – |
Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-10 to 2026-11-16. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two
times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.
Other devices
| Device | Backend | dtype | ms / call | Peak memory | Output vs reference | Input |
|---|---|---|---|---|---|---|
| Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-14 | Original | fp32 | 14.6 | 2,148 MiB process | – | 224x224 |
| Portable | fp16 | 9.96 | 871 MiB process | matches | 224x224 | |
| Portable | fp32 | 10.9 | 865 MiB process | matches | 224x224 | |
| Kernels | fp16 | not implemented | – | 224x224 | ||
CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.
Licence and weights
Commercial use allowed
- Code
- Apache-2.0
- Weights
- Apache-2.0 source
- Training data
-
- ImageNet-21k / ImageNet-22k (full ImageNet, fall 2011 release) noncommercial, non-commercial
Our reading of the licences, not legal advice.
Weights
convnext_tiny.fb_in22k.safetensors 178 MB· hosted on Hugging Facefastforks downloads and checks these files on first use (fastforks download convnext_tiny.fb_in22k).
Accuracy
The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.
ImageNet-1k val
kNN top-1 % (k = 20, 10000 labelled / 5000 test images)
| Run | dtype | top-1 % ↑ | Δ top-1 % vs fp32 original | images | ms / image |
|---|---|---|---|---|---|
| Original fp32 | fp32 | 66.80 | reference | 5000 | 1.37 |
| Original fp16 | fp16 | 66.88 | +0.08 | 5000 | 0.769 |
| Portable fp32 | not measured | ||||
| Portable | fp16 | 66.82 | +0.02 | 5000 | 0.451 |
| Kernels | fp16 | 66.84 | +0.04 | 5000 | 0.334 |
Two disjoint seeded subsets of the val folders; each image shorter side 256 (bicubic), centre crop 224, the CLS / pooled feature; cosine kNN, vote weighted by similarity. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-10; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 8b2f2df5, 2026-11-10.
Closeness to the original
Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 8.1e-05 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.6 % and p99 <= 9.4 % of the reference's median |value| (absolute error of the output last_hidden_state).
| Check | Tolerance | Measured |
|---|---|---|
| CPU fp32, max |diff| | ≤ 0.000081 (relative) | not recorded |
| GPU default dtypes, mean |diff| | ≤ 0.016 (relative) | 0.0066 (relative) |
| GPU default dtypes, p99 |diff| | ≤ 0.094 (relative) | 0.03812 (relative) |
Recorded for convnext_tiny.fb_in22k on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-13, with these kernels: fvm_convfvm_seg
Details
ConvNeXt (Liu et al. 2022) as a feature backbone: Meta's ImageNet-22k checkpoints in timm's format (tiny, small, base, large). last_hidden_state = the stride-32 stage-4 map as tokens (no prefix token, un-normalised, as timm's forward_features / features_only and HF ConvNextModel); pooler_output = the head LayerNorm of their mean (timm's pre-logits embedding).
- Input / output
- Input: one RGB image. Output: dense patch features and a pooled image feature.
- Input size
- any H x W >= 1, resized or padded inside the model; tested up to 3000x4000 (1,905 MiB allocated)
- Outputs
last_hidden_state,pooler_output- Checkpoint
arch="convnext_tiny.fb_in22k"- CUDA kernels
fvm_convfused convolutions for image classifiersfvm_segfused kernels for Mask2Former and OneFormer
- Apple (MPS)
- fp32, fp16
Limitations
- Only the last stage's map is returned: the maps of stages 1-3 (strides 4, 8, 16; timm `features_only=True`) have other sizes than the token grid of this task's outputs.
- kernels replays a CUDA graph per small input shape (up to 2^20 input pixels per call), captured on the shape's second call, at most 4 per model; `model.release_graphs()` frees them.
Full reference, generated from the repository
Meta's ConvNeXt (tiny, small, base, large) trained on ImageNet-22k, without the classifier: the last stage's feature map as tokens and a pooled image embedding. Apache-2.0 weights.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
convnext_tiny.fb_in22k | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 3.89 5.91 3.26 1.10 2.08 0.726 | 151 / 170 194 / 214 93 / 108 88 / 110 148 / 170 183 / 200 |
convnext_small.fb_in22k | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 7.51 11.2 5.92 1.79 3.36 1.16 | 234 / 252 318 / 336 134 / 150 130 / 150 210 / 230 277 / 292 |
convnext_base.fb_in22k | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 7.68 11.2 5.80 1.96 3.21 1.51 | 386 / 398 537 / 544 209 / 214 204 / 216 321 / 334 480 / 494 |
convnext_large.fb_in22k | synthetic (224x224) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 10.5 11.5 5.82 3.23 3.34 2.50 | 822 / 834 1160 / 1180 420 / 430 413 / 434 639 / 650 983 / 1006 |
Details
- Original code: https://github.com/huggingface/pytorch-image-models at
0df212b369a5 - Code licence: Apache-2.0 (permissive); files:
- Weights licence: Apache-2.0
- Licence evidence: https://huggingface.co/timm/convnext_base.fb_in22k/blob/9afd4146adabf1b6e861d0f1ac2e7baa7c8dd1b7/README.md
- Training data: ImageNet-21k / ImageNet-22k (full ImageNet, fall 2011 release)
- Kernel libraries:
fvm_conv,fvm_seg - Variants:
convnext_tiny.fb_in22k,convnext_small.fb_in22k,convnext_base.fb_in22k,convnext_large.fb_in22k - Outputs: last_hidden_state, pooler_output
- Input size: any H x W >= 1, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
convnext_tiny.fb_in22k3000x4000 (1.9 GiB peak allocated, 2.4 reserved);convnext_small.fb_in22k3000x4000 (1.9 GiB peak allocated, 2.5 reserved);convnext_base.fb_in22k3000x4000 (2.5 GiB peak allocated, 3.6 reserved);convnext_large.fb_in22k3000x4000 (3.8 GiB peak allocated, 5.2 reserved) convnext_tiny.fb_in22k224x224, first call: portable + torch.compile 12 s (cold Inductor cache), TensorRT fp16 34 s (ONNX + engine).convnext_small.fb_in22k224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 35 s (ONNX + engine).convnext_base.fb_in22k224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 30 s (ONNX + engine).convnext_large.fb_in22k224x224, first call: portable + torch.compile 10 s (cold Inductor cache), TensorRT fp16 47 s (ONNX + engine).convnext_tiny.fb_in22koriginal (fp32) 224x224, original 224x224: timm num_classes=0 (pooled, head norm)convnext_small.fb_in22koriginal (fp32) 224x224, original 224x224: timm num_classes=0 (pooled, head norm)convnext_base.fb_in22koriginal (fp32) 224x224, original 224x224: timm num_classes=0 (pooled, head norm)convnext_large.fb_in22koriginal (fp32) 224x224, original 224x224: timm num_classes=0 (pooled, head norm)convnext_tiny.fb_in22kload / first call: original 0.4 s / 0.275 s, portable 1.66 s / 0.23 s, kernels 1.57 s / 0.221 sconvnext_small.fb_in22kload / first call: original 0.65 s / 0.014 s, portable 1.93 s / 0.209 s, kernels 1.73 s / 0.18 sconvnext_base.fb_in22kload / first call: original 0.9 s / 0.019 s, portable 2.18 s / 0.237 s, kernels 2.05 s / 0.244 sconvnext_large.fb_in22kload / first call: original 2.17 s / 0.02 s, portable 2.71 s / 0.238 s, kernels 2.78 s / 0.232 s- Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5
Measured
Two disjoint seeded subsets of the val folders; each image shorter side 256 (bicubic), centre crop 224, the CLS / pooled feature; cosine kNN, vote weighted by similarity. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
convnext_tiny.fb_in22k | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 66.80 | 66.88 | 66.82 | 66.84 | |
convnext_small.fb_in22k | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 71.68 | 71.68 | 71.72 | 71.72 | |
convnext_base.fb_in22k | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 73.04 | 73.04 | 73.08 | 73.04 | |
convnext_large.fb_in22k | ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 75.26 | 75.22 | 75.28 | 75.24 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 8b2f2df5, 2026-11-10.
Parity
CPU, float32: max |diff| <= 8.1e-05 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.6 % and p99 <= 9.4 % of the reference's median |value| (absolute error of the output last_hidden_state).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
convnext_tiny.fb_in22k | 0.0066 / 0.03812 | RTX 5060 Ti, torch 2.14.1+cu130 | fvm_conv@f1d064e539e1:sm_120, fvm_seg@739693dff3b4:sm_120 |
convnext_small.fb_in22k | 0.00726 / 0.04292 | RTX 5060 Ti, torch 2.14.1+cu130 | fvm_conv@f1d064e539e1:sm_120, fvm_seg@739693dff3b4:sm_120 |
convnext_base.fb_in22k | 0.0104 / 0.06216 | RTX 5060 Ti, torch 2.14.1+cu130 | fvm_conv@f1d064e539e1:sm_120, fvm_seg@739693dff3b4:sm_120 |
convnext_large.fb_in22k | 0.00787 / 0.04251 | RTX 5060 Ti, torch 2.14.1+cu130 | fvm_conv@f1d064e539e1:sm_120, fvm_seg@739693dff3b4:sm_120 |
Limitations
- Only the last stage's map is returned: the maps of stages 1-3 (strides 4, 8, 16; timm
features_only=True) have other sizes than the token grid of this task's outputs. - kernels replays a CUDA graph per small input shape (up to 2^20 input pixels per call), captured on the shape's second call, at most 4 per model;
model.release_graphs()frees them.
Example output
One fixed input through this checkpoint, the same for every model of the task (kernels · float16 · NVIDIA GeForce RTX 5060 Ti).
Input: Chelsea the cat, Stéfan van der Walt (CC0).