Other GPUs

All numbers and reference checks in these docs come from one RTX 5060 Ti 16 GB (sm_120). This page explains what runs on other hardware, how to check your setup, and how to measure your GPU and send us the numbers.

What runs where

portable is plain PyTorch and runs wherever PyTorch runs (CPU, any CUDA GPU, MPS). kernels adds our CUDA kernels. It needs all of the following:

• Linux on x86-64 with glibc 2.34 or newer. aarch64 hosts (Jetson, Grace, DGX Spark) run portable only.
• An NVIDIA GPU with compute capability 8.0 or newer.
• An NVIDIA driver for CUDA 12.8 or newer (driver 570 or newer). The kernels are built with CUDA 12.8, those of GREAT-IGEV, IGEV++, MoCha-Stereo, S2M2 and fvm_conv (all classification families, DINOv3-ConvNeXt, SegFormer) with CUDA 13.0 (driver 580 or newer). These libraries (fvm_conv, igevpp, mocha, s2m2_cutlass) also ship their CUDA 12.8 build, which drivers 570-579 load automatically (fvm_conv's is slower: ResNet-50 core 0.76 instead of 0.64 ms).
• cuDNN 9 (the copy torch ships): most stereo families build fp16-accumulate cuDNN plans through its backend API.
• One CUDA device per process for kernels.

Each kernel library ships native code (SASS) for sm_80, 86, 89, 90, 100 and 120, plus PTX for compute_80. A GPU uses the exact SASS if there is one. Otherwise it uses SASS with the same major version and the highest lower minor, otherwise the PTX, which the driver compiles on the first load.

"Built but not tested" means the kernels compile for that target, but nobody has checked them against the reference outputs on such a GPU. If kernels gives wrong results, or a different speed-up than Benchmarks, please report it. portable with fp16 is the same PyTorch code on every GPU.

Precision: on CUDA, the default dtype=None uses each model's tested reduced precision (float16, or bfloat16 for DEFOM). GPUs before compute capability 8.0 have no fast bfloat16, so DEFOM computes in float32 there automatically.

mode="auto" (the default) picks kernels when it can load and falls back to portable otherwise. It logs the reason once per load: at WARNING (visible with default logging) when kernels fails on a CUDA device, else at INFO. mode="kernels" never falls back: it raises BackendUnavailableError / KernelUnavailableError with the reason and a remedy.

What each family runs on

Per family (families with the same support share a row; from the model cards; variants of a family share it). CPU: portable, float32 only. kernels runs only on CUDA and on MPS (where listed); all other pairs run portable. fp32 / fp16 / bf16 = float32 / float16 / bfloat16. "MPS portable (validated)" lists the dtypes checked on Apple silicon, the first is the default; - = not validated there (device="mps" still tries, and an op MPS lacks raises).

On MPS (Apple silicon), portable works for every validated family and mode="auto" picks the Metal kernels where the card lists mps_kernels (card.kernel_libs_on(device) returns the Metal libraries there, the CUDA ones elsewhere). The Metal kernels compile on the device at first use. They were written and tested on an M1 Pro only; the changes of 2026-11-04 (mps.adapt on all Metal runners, win_rpe_attn sentinel, kernel input guards, rectify in float64 on CPU) still need a check on a Mac.

Kernel libraries per family

Each family's kernels backend needs these libraries; fastforks env shows whether a binary exists for your GPU.

Check your setup

fastforks env

This prints the fastforks, Python, torch, CUDA, cuDNN and driver versions, and the GPU with its compute capability. For each kernel library it shows which binary your GPU would use and whether it is already downloaded. Example on the tested machine (RTX 5060 Ti, 2026-10-27):

fastforks              0.1.0
python                 3.12.15 (linux x86_64)
torch                  2.14.1+cu130 (CUDA 13.0, cuDNN 92400)
driver                 supports CUDA 13.2
gpu                    NVIDIA GeForce RTX 5060 Ti, sm_120
kernels cfs            sm_120, local
kernels crestereo      sm_120, local
kernels cudnn_hacc     no binaries: cuDNN plans via the cuDNN backend API (cuDNN 9.24.0)
kernels cutlass_conv   sm_120, local
kernels fastbb         sm_120, local
kernels fs             sm_120, local
kernels fs_cutlass     sm_120, local
kernels hitnet         sm_120, local
kernels hitnet_conv    sm_120, local
kernels igev           sm_120, local
kernels raft           sm_120, local
kernels s2m2           sm_120, local
kernels s2m2_cutlass   sm_120, local

Possible states of a kernel library:

Which backend a loaded model uses:

import logging

from fastforks import stereo

logging.basicConfig(level=logging.INFO)  # shows every backend choice, e.g. "kernels unavailable (...); using portable"
model = stereo.load("las2-s")
print(model.backend.mode)  # "kernels" or "portable"
stereo.load("las2-s", mode="kernels")  # raises with the reason instead of falling back

Offline machines: download weights and kernels on a machine with internet, copy the cache, and point FASTFORKS_CACHE at the copy.

fastforks download las2-s depth-anything-v2-s --kernels --all-archs   # --all-archs: binaries for every target
# copy the cache folder (see FASTFORKS_CACHE) to the offline machine, then there:
export FASTFORKS_CACHE=/path/to/copy FASTFORKS_OFFLINE=1

Common errors

Measure your GPU

fastforks bench times one model on a synthetic image (a pair for stereo). It runs 4 warm-up calls, then prints the median of --runs calls and the peak memory of the torch allocator:

fastforks bench las2-s --mode kernels --size 375x1242 --runs 20
fastforks bench las2-s --mode portable --size 375x1242 --runs 20

Output of the two commands on the tested machine (2026-11-05):

las2-s@1 375x1242 kernels float16: 4.0 ms (median of 20), peak 115 MiB allocated / 326 MiB reserved
las2-s@1 375x1242 portable float16: 20.5 ms (median of 20), peak 302 MiB allocated / 330 MiB reserved

To measure the models of the Benchmarks tables in both modes and save everything to one file:

gpu=$(nvidia-smi --query-gpu=name --format=csv,noheader | head -1 | tr ' ' '_')
out=fastforks_bench_$gpu.txt
fastforks env > "$out"
for m in las2-s hitnet-eth3d coex-sceneflow s2m2-s igev-stereo-sceneflow crestereo-eth3d depth-anything-v2-s; do
  for mode in kernels portable; do
    fastforks bench "$m" --mode "$mode" --size 375x1242 --runs 20 2>&1 | tee -a "$out"
  done
done

Close other GPU programs first. Laptop GPUs and GPUs without a fixed clock vary by 5-10 % between runs, so run the loop twice and report both runs.

Check that kernels and portable agree on your GPU:

import numpy as np

from fastforks import stereo

rng = np.random.default_rng(0)
left = rng.integers(0, 256, (375, 1242, 3), dtype=np.uint8)
right = np.roll(left, -8, axis=1)
a = stereo.load("las2-s", mode="kernels")(left, right).disparity
b = stereo.load("las2-s", mode="portable")(left, right).disparity
print("max |kernels - portable| px:", (a - b).abs().max().item(), "mean:", (a - b).abs().mean().item())

On the tested machine this prints max |kernels - portable| px: 4.794837951660156 mean: 0.1790103018283844 (2026-10-27). Random images have no real structure, so single pixels can differ by several pixels of disparity: compare the mean. The GPU reference tests compare both modes with one stored reference output on a synthetic pair (tests/synth.py), by the mean and the 99th percentile of the difference.

Contributing numbers

Open an issue at github.com/kleinicke/fastforks titled "Numbers: " and attach:

1. the fastforks env output,
2. the bench file from the loop above (both runs),
3. the output of the agreement snippet,
4. anything unusual: errors, a slow first load (PTX JIT), clocks locked or not, laptop or desktop.

Numbers from untested architectures (everything except sm_120) are the most useful. If a kernels load fails, the full error message and fastforks env are enough.