FoundationStereo · fastforks

NVIDIA's zero-shot stereo foundation model. Non-commercial weights.

Commercial use: no (code: LicenseRef-NVIDIA-Source-Code-License-NC AND Apache-2.0 (noncommercial); weights: LicenseRef-NVIDIA-Source-Code-License-NC).

variant	input	backend	time (ms)	peak memory (MiB, allocated / reserved)
`foundationstereo-vit-l`	Middlebury 2014 half (994x1482)	original (fp32) original portable kernels	9228 4237 2990 1513	11323 / 11632 8286 / 8750 3141 / 3436 2818 / 3300
`foundationstereo-vit-s`	Middlebury 2014 half (994x1482)	original (fp32) original portable kernels portable + torch.compile TensorRT fp16	7129 3609 2514 1120 2044 1196	9991 / 15144 6987 / 7280 2493 / 2606 2169 / 2470 2822 / 3646 2570 (engine)

Details

Original code: https://github.com/NVlabs/FoundationStereo at 6e8806816b53
Code licence: LicenseRef-NVIDIA-Source-Code-License-NC AND Apache-2.0 (noncommercial); files: LICENSE
Weights licence: LicenseRef-NVIDIA-Source-Code-License-NC
Licence evidence: https://github.com/NVlabs/FoundationStereo/blob/6e8806816b533e4d13ddbb95ffa907b797060a62/readme.md#L158
Training data: Foundation Stereo Dataset (FSD, synthetic, NVIDIA) plus SceneFlow, Falling Things, Sintel, CREStereo, InStereo2K and others; no per-dataset terms checked
Licence note: Code and weights are under the NVIDIA Source Code License-NC (non-commercial research use only). Our kernels for this family count as derived work and inherit the NC limit; the family ships in fastforks-nc.
Options: iters = 32 (>= 1)
Kernel libraries: fs, fs_cutlass, cudnn_hacc, fastbb
Variants: foundationstereo-vit-l, foundationstereo-vit-s
Outputs: disparity
Input size: any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): foundationstereo-vit-l 1988x2880 (7.1 GiB peak allocated, 11.4 reserved); foundationstereo-vit-s 1988x2880 (6.5 GiB peak allocated, 10.7 reserved)
foundationstereo-vit-s 994x1482, first call: portable + torch.compile 212 s (cold Inductor cache), TensorRT fp16 545 s (ONNX + engine).
foundationstereo-vit-l load / first call: original 10.86 s / 2.563 s (375x1242), portable 1.98 s / 2.99† s, kernels 1.17† s / 1.517† s
foundationstereo-vit-s load / first call: original 1.44 s / 1.058 s (375x1242), portable 1.03 s / 2.514† s, kernels 0.33† s / 1.12† s
† measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

Original = the original code named under Details, same checkpoint and protocol.

variant	dataset	original fp32	original fp16	portable	kernels	note
`foundationstereo-vit-l`	ETH3D two-view train, bad > 1 px % / EPE	0.65 / 0.168	0.65 / 0.168	0.65 / 0.168	0.65 / 0.168	possibly in the training data
`foundationstereo-vit-l`	Middlebury 2014 trainingH, bad > 2 px % / EPE	2.48 / 0.562	2.48 / 0.561	2.47 / 0.561	2.48 / 0.561	possibly in the training data
`foundationstereo-vit-s`	ETH3D two-view train, bad > 1 px % / EPE	1.10 / 0.202	1.10 / 0.202	1.10 / 0.202	1.10 / 0.202	possibly in the training data
`foundationstereo-vit-s`	Middlebury 2014 trainingH, bad > 2 px % / EPE	3.32 / 0.679	3.32 / 0.681	3.32 / 0.680	3.30 / 0.680	possibly in the training data

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 468fba10, 2026-11-05.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 px and p99 <= 0.25 px.

variant	kernels error vs original, mean / p99	GPU, torch	kernels used
`foundationstereo-vit-l`	0.00035 / 0.00121	RTX 5060 Ti, torch 2.14.1+cu130	`cudnn_hacc@cudnn92400`, `fastbb@7d058322634d:sm_120`, `fs@f6f8de2cca1e:sm_120`, `fs_cutlass@de5f3c10aa1b:sm_120`
`foundationstereo-vit-s`	0.00029 / 0.0009	RTX 5060 Ti, torch 2.14.1+cu130	`cudnn_hacc@cudnn92400`, `fastbb@7d058322634d:sm_120`, `fs@f6f8de2cca1e:sm_120`, `fs_cutlass@de5f3c10aa1b:sm_120`

Limitations

ONNX export needs torch >= 2.14 (older exporters fail on this network).