DEFOM-Stereo · fastforks

A stereo network from the Insta360 Research Team that uses Depth Anything V2 features and an initial depth prior. Non-commercial weights.

Commercial use: no (weights: CC-BY-NC-4.0).

variant	input	backend	time (ms)	peak memory (MiB, allocated / reserved)
`defom-stereo-vitl-sceneflow`	Middlebury 2014 half (994x1482)	original (fp32) original portable kernels portable + torch.compile TensorRT fp16	4175 2816 1531 659 1002 fails: TensorRT bug: Myelin internal error	10896 / 11072 8639 / 8766 4362 / 4838 3154 / 3566 3021 / 3292 –

Details

Original code: https://github.com/Insta360-Research-Team/DEFOM-Stereo at 5b27591d8395
Code licence: MIT (permissive); files: LICENSE
Weights licence: CC-BY-NC-4.0
Licence evidence: https://github.com/Insta360-Research-Team/DEFOM-Stereo/blob/5b27591d83956e5d329ff66704271a9e68b1c75f/README.md
Training data: Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa)
Licence note: The weights embed Depth Anything V2 ViT-L weights released under CC-BY-NC-4.0, so they are non-commercial even though the code is MIT.
Options: iters = 32 (>= 1), scale_iters = 8 (>= 0)
Kernel libraries: igev, cutlass_conv, cudnn_hacc, fastbb
Variants: defom-stereo-vitl-sceneflow
Outputs: disparity
Input size: any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): defom-stereo-vitl-sceneflow 1988x2880 (12.0 GiB peak allocated, 14.5 reserved)
defom-stereo-vitl-sceneflow 994x1482, first call: portable + torch.compile 122 s (cold Inductor cache).
defom-stereo-vitl-sceneflow original 994x1482: the authors' mixed-precision option (float16)
defom-stereo-vitl-sceneflow load / first call: original 4.24 s / 2.83 s, portable 2.31 s / 1.534† s, kernels 1.18† s / 0.663† s
† measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

Original = the original code named under Details, same checkpoint and protocol.

variant	dataset	original fp32	original fp16	portable	kernels	note
`defom-stereo-vitl-sceneflow`	ETH3D two-view train, bad > 1 px % / EPE	2.40 / 0.337	99.92 / 13.132 (NaN / inf output)	2.37 / 0.356	2.44 / 0.341
`defom-stereo-vitl-sceneflow`	Middlebury 2014 trainingH, bad > 2 px % / EPE	6.03 / 0.837	100.00 / 71.272 (NaN / inf output)	6.15 / 0.845	6.02 / 0.834

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 468fba10, 2026-11-05.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, portable bfloat16 and kernels float16 (the defaults): mean |diff| <= 0.02 px and p99 <= 0.25 px.

variant	kernels error vs original, mean / p99	GPU, torch	kernels used
`defom-stereo-vitl-sceneflow`	0.00143 / 0.00808	RTX 5060 Ti, torch 2.14.1+cu130	`cudnn_hacc@cudnn92400`, `cutlass_conv@988138e042cb:sm_120`, `fastbb@7d058322634d:sm_120`, `igev@f075827d68c6:sm_120`

Limitations

Only the ViT-L SceneFlow checkpoint is included (ViT-S / ViT-B and the Middlebury / ETH3D / KITTI fine-tunes are not).
Portable is faster than the original code but allocates more memory (about 1.25x at 994x1482); kernels is several times faster and needs a fraction of the memory.