HITNet
Google Research's tile-based stereo model; the authors' ETH3D and Middlebury versions, ported to PyTorch. Commercially usable.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
hitnet-eth3d | Middlebury 2014 half (994x1482) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 234 182 67.8 23.2 27.7 31.1 | 2961 / – 2433 / – 860 / 1090 647 / 1248 719 / 728 836 (engine) |
hitnet-middlebury-d400 | Middlebury 2014 half (994x1482) | original (fp32) original portable kernels | 637 483 176 52.7 | 7157 / – 6831 / – 1744 / 2414 1263 / 2368 |
Details
- Original code: https://github.com/google-research/google-research at
f200f2b89d29 - Code licence: Apache-2.0 (permissive); files:
LICENSE - Weights licence: Apache-2.0 (assumed from code)
- Licence evidence: https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE
- Training data: not stated by the authors
hitnet-eth3d: ETH3D two-view stereo (fine-tuned on the ETH3D training pairs (paper); any pre-training data is not stated by the authors)hitnet-middlebury-d400: Middlebury stereo (2014) (trained for half-resolution Middlebury images (README); any pre-training data is not stated by the authors)
- Licence note: The authors state no weights licence; the checkpoints are fetched from the PINTO model-zoo mirror (a third-party host), not from Google, and the code licence is assumed to cover them.
- Options:
cuda_graphs= True (speed only) - Kernel libraries:
hitnet,hitnet_conv - Variants:
hitnet-eth3d,hitnet-middlebury-d400 - Outputs: disparity
- Input size: any H x W >= 8, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
hitnet-eth3d3000x4000 (6.4 GiB peak allocated, 10.8 reserved);hitnet-middlebury-d4001988x2880 (6.4 GiB peak allocated, 10.1 reserved) hitnet-eth3d994x1482, first call: portable + torch.compile 39 s (cold Inductor cache), TensorRT fp16 136 s (ONNX + engine).hitnet-eth3doriginal (fp32) 994x1482: TensorFlow 2.21.0 frozen graph eth3d.pb, float32hitnet-eth3doriginal 994x1482: TensorFlow 2.21.0 frozen graph eth3d.pb, auto_mixed_precision graph rewrite (float16)hitnet-middlebury-d400original (fp32) 994x1482: TensorFlow 2.21.0 frozen graph middlebury_d400.pb, float32hitnet-middlebury-d400original 994x1482: TensorFlow 2.21.0 frozen graph middlebury_d400.pb, auto_mixed_precision graph rewrite (float16)hitnet-eth3dload / first call: original 0.31 s / 2.061 s, portable 0.09 s / 0.078† s, kernels 0.07† s / 0.026† shitnet-middlebury-d400load / first call: original 0.57 s / 3.512 s, portable 0.13 s / 0.177† s, kernels 0.1† s / 0.053† s- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
hitnet-eth3d | ETH3D two-view train, bad > 1 px % / EPE | 0.94 / 0.164 | 0.93 / 0.165 | 0.93 / 0.164 | 0.93 / 0.164 | trained on this dataset's public images; not a zero-shot result |
hitnet-eth3d | Middlebury 2014 trainingH, bad > 2 px % / EPE | 17.26 / 8.332 | 17.30 / 8.356 | 17.25 / 8.340 | 17.27 / 8.297 | |
hitnet-middlebury-d400 | ETH3D two-view train, bad > 1 px % / EPE | 7.02 / 0.885 | 7.00 / 0.880 | 7.01 / 0.899 | 7.07 / 0.875 | |
hitnet-middlebury-d400 | Middlebury 2014 trainingH, bad > 2 px % / EPE | 7.50 / 1.180 | 7.51 / 1.185 | 7.54 / 1.187 | 7.51 / 1.168 | trained on this dataset's public images; not a zero-shot result |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 468fba10, 2026-11-05.
Parity
CPU, float32: max |diff| <= 0.02 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.15 px and p99 <= 1.0 px. On real pairs up to 720x1280 the limits are mean 0.3 / p99 6.0 (precision-sensitive there, see Limitations).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
hitnet-eth3d | 0.00343 / 0.02548 | RTX 5060 Ti, torch 2.14.1+cu130 | hitnet@45145b49dcb1:sm_120, hitnet_conv@e5637e8a93cf:sm_120 |
hitnet-middlebury-d400 | 0.1139 / 0.74705 | RTX 5060 Ti, torch 2.14.1+cu130 | hitnet@45145b49dcb1:sm_120, hitnet_conv@e5637e8a93cf:sm_120 |
Limitations
- CUDA output (float16, kernels, float32 with PyTorch's default TF32 convolutions) differs from the CPU by ~0.2 px mean / 4 px p99 at 720x1280: the argmin flips on near-equal tile costs. Full float32 (
torch.backends.cudnn.allow_tf32 = False) matches the CPU to 0.00002 px. - The weights are the authors' frozen TensorFlow graphs read without TensorFlow; the FlyingThings and Middlebury d160 / d288 graphs are not included.
- The original code is TensorFlow and was not timed.