Models · Optical flow

NeuFlow v2

NeuFlow v2 (Northeastern University, 2024): a small and fast optical-flow network built for edge devices: a light CNN backbone, global matching at 1/16 and a few cheap refinements at 1/8. Apache-2.0 code and weights.

experimental Fork of neufieldrobotics/NeuFlow_v2@204b5e3 · backends: portable, kernels

Checkpoint

Paper 2024 Added in 0.2.0 Evaluated with 0.2.0

Kernels, RTX 5060 Ti 16 GB 59.1 ms 1080x1920, fp16
vs the faster original 1.3× original fp16: 75.6 ms
Peak memory 790 MiB original: 4,367 MiB
Accuracy vs original fp32 −0.009 kernels EPE, KITTI 2015 train (1.568 original)

Use it

pip install fastforks

Lean install with only what this model needs: pip install fastforks-core

from fastforks import flow

model = flow.load("neuflow-v2")  # kernels on a supported GPU, else portable PyTorch
result = model(frame1, frame2)
result.flow
result.backend  # what ran: mode, device, dtype, kernels
fastforks bench neuflow-v2

Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.

Speed and memory

Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.

BackenddtypeTime per call (ms)vs original fp16Peak memory
allocated / reserved
Load (s)First call (s)
Original fp32upstream NeuFlow, iters_s8=8fp32 1710.44×8,687 MiB/9,216 MiB2.10.54
Original fp16the authors' mixed-precision option (float16); upstream NeuFlow, iters_s8=8, infer.py recipe: conv-bn folded, model.half()fp16 75.6–4,367 MiB/4,556 MiB0.20.22
Portable fp32fp32not measured ––––
Portablefp16 71.21.1×2,366 MiB/2,398 MiB0.30.07
Kernelsfp16 59.11.3×790 MiB/896 MiB0.90.06
Portable + torch.compile 91.40.83×4,529 MiB/4,558 MiB29 (build)–
TensorRT fp16fp16 65.01.2×8,532 MiBenginenot comparable104 (build)–

Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-11 to 2026-11-15. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.

Other devices

DeviceBackenddtypems / callPeak memoryOutput vs referenceInput
Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-14Originalfp32out of memory–1080x1920
Portablefp164552 11.9 GiB processmatches1080x1920
Portablefp321039 13.0 GiB processmatches1080x1920
Kernelsfp16not implemented–1080x1920

CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.

Licence and weights

Commercial use allowed

Code
Apache-2.0 LICENSE
Weights
Apache-2.0 source
Training data

Our reading of the licences, not legal advice.

Weights

neuflow_mixed.pth 36.2 MB· hosted on GitHub

fastforks downloads and checks these files on first use (fastforks download neuflow-v2).

Accuracy

The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.

KITTI 2015 train

EPE / Fl-all %trained on this dataset’s public images: not zero-shot

RundtypeEPE ↓Fl-all % ↓Δ EPE vs fp32 originalpairsms / pair
Original fp32fp321.5685.27 reference 200 27.8
Original fp16fp161.5885.28 −0.020 200 12.6
Portable fp32fp321.5685.27 +0.000 200 27.4
Portablefp161.5795.28 −0.011 200 14.4
Kernelsfp161.5775.28 −0.009 200 12.4

Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-11; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6f0a70e1, 2026-11-17; portable: RTX 5060 Ti, torch 2.14.1+cu130, fastforks c4a67979, 2026-11-11; kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks f3875362, 2026-11-11.

Closeness to the original

Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0014 and p99 <= 0.0067 (absolute error of the output flow).

CheckToleranceMeasured
CPU fp32, max |diff|≤ 0.0001 not recorded
GPU default dtypes, mean |diff|≤ 0.0014 0.00075
GPU default dtypes, p99 |diff|≤ 0.0067 0.00425

Recorded for neuflow-v2 on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-12, with these kernels: raft_flow

Details

NeuFlow v2 (Zhang, Gupta, Singh & Jiang 2024): a light optical-flow network for robots and edge devices. A plain convolutional backbone on the image averaged down by 2, 4 and 8, cross-attention and global matching at 1/16 resolution for the initial flow (large motions), then one refinement at 1/16 and 8 at 1/8 with a 9 x 9 correlation lookup and plain 3 x 3 convs (no GRU), convex upsampling. The authors' released weights; the network is ported from neufieldrobotics/NeuFlow_v2. `neuflow-v2` is the authors' mixed-dataset checkpoint (their default), `-things` the FlyingThings3D one (zero-shot elsewhere), `-sintel` the Sintel fine-tune stage.

Input / output
Input: two RGB images (consecutive frames). Output: per-pixel motion from the first to the second (dx, dy in pixels).
Input size
any H x W >= 32, padded to a multiple of 16; tested up to 1024x2048 (8,400 MiB allocated)
Options
  • iters=8>= 1· changes the output
Outputs
flow
CUDA kernels
  • raft_flowRAFT optical flow kernels
Apple (MPS)
fp32, fp16

Limitations

  • Portable at 375 x 1242 runs 15.1 ms vs 12.2 ms for the authors' fp16 lookup (24 % slower, launch-bound; at 1080 x 1920 it is faster, 71.4 vs 75.8 ms): lookup, matching and upsampling stay float32 on purpose (lower KITTI EPE); fp16 lookup or channels_last did not help.
  • Kernels (raft_flow) run 12.0 ms / 205 MB, faster than the original, but the binary is not on Hugging Face yet; it is built locally and found through FASTFORKS_KERNEL_PATH.
Full reference, generated from the repository

NeuFlow v2 (Northeastern University, 2024): a small and fast optical-flow network built for edge devices: a light CNN backbone, global matching at 1/16 and a few cheap refinements at 1/8. Apache-2.0 code and weights.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
neuflow-v2synthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
171
75.6
71.2
59.1
91.4
65.0
8687 / 9216
4367 / 4556
2366 / 2398
790 / 896
4529 / 4558
8710 / 8748
neuflow-v2-sintelsynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
171
75.7
71.3
58.9
91.4
64.9
8694 / 9020
4367 / 4576
2366 / 2398
790 / 896
4529 / 4558
8710 / 8748
neuflow-v2-thingssynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
172
75.8
71.4
58.9
91.5
65.0
8694 / 9040
4374 / 4720
2366 / 2398
779 / 896
4529 / 4558
8710 / 8748
Details
  • Original code: https://github.com/neufieldrobotics/NeuFlow_v2 at 204b5e374446
  • Code licence: Apache-2.0 (permissive); files: LICENSE
  • Weights licence: Apache-2.0
  • Licence evidence: https://huggingface.co/Study-is-happy/neuflow-v2/blob/79ae2f4589456d4d369aaef4955c7db5918c3436/README.md
  • Training data: Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (README: "trained with multiple datasets" (not listed); the mixed stages of upstream's data_utils/datasets.py use FlyingThings3D (part of SceneFlow), Sintel, KITTI 2015, HD1K and, in one stage, VIPER)
    • neuflow-v2-sintel: Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (FlyingThings3D (part of SceneFlow), then upstream's 'sintel' stage: Sintel, KITTI 2015, HD1K and FlyingThings3D)
    • neuflow-v2-things: Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa) (FlyingThings3D (part of SceneFlow), upstream's 'things' stage)
  • Options: iters = 8 (>= 1)
  • Kernel libraries: raft_flow
  • Variants: neuflow-v2, neuflow-v2-sintel, neuflow-v2-things
  • Outputs: flow
  • Input size: any H x W >= 32, padded to a multiple of 16
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): neuflow-v2 1024x2048 (8.2 GiB peak allocated, 9.0 reserved); neuflow-v2-sintel 1024x2048 (8.2 GiB peak allocated, 9.0 reserved); neuflow-v2-things 1024x2048 (8.2 GiB peak allocated, 9.0 reserved)
  • neuflow-v2 1080x1920, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 104 s (ONNX + engine).
  • neuflow-v2-sintel 1080x1920, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 102 s (ONNX + engine).
  • neuflow-v2-things 1080x1920, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 104 s (ONNX + engine).
  • neuflow-v2 original (fp32) 1080x1920: upstream NeuFlow, iters_s8=8
  • neuflow-v2 original 1080x1920: the authors' mixed-precision option (float16); upstream NeuFlow, iters_s8=8, infer.py recipe: conv-bn folded, model.half()
  • neuflow-v2-sintel original (fp32) 1080x1920: upstream NeuFlow, iters_s8=8
  • neuflow-v2-sintel original 1080x1920: the authors' mixed-precision option (float16); upstream NeuFlow, iters_s8=8, infer.py recipe: conv-bn folded, model.half()
  • neuflow-v2-things original (fp32) 1080x1920: upstream NeuFlow, iters_s8=8
  • neuflow-v2-things original 1080x1920: the authors' mixed-precision option (float16); upstream NeuFlow, iters_s8=8, infer.py recipe: conv-bn folded, model.half()
  • neuflow-v2 load / first call: original 0.2 s / 0.223 s, portable 0.26 s / 0.072 s, kernels 0.96 s / 0.179 s (375x1242)
  • neuflow-v2-sintel load / first call: original 0.17 s / 0.076 s, portable 1.29 s / 0.806 s (375x1242), kernels 0.95 s / 0.178 s (375x1242)
  • neuflow-v2-things load / first call: original 0.17 s / 0.076 s, portable 0.38 s / 0.072 s, kernels 0.26 s / 0.059 s
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5

Measured

Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portableportable fp32kernelsnote
neuflow-v2KITTI 2015 train, EPE / Fl-all %1.568 / 5.271.588 / 5.281.579 / 5.281.568 / 5.271.577 / 5.28trained on this dataset's public images; not a zero-shot result
neuflow-v2-sintelKITTI 2015 train, EPE / Fl-all %1.566 / 5.611.573 / 5.611.567 / 5.611.566 / 5.611.566 / 5.61trained on this dataset's public images; not a zero-shot result
neuflow-v2-thingsKITTI 2015 train, EPE / Fl-all %4.099 / 15.024.098 / 15.014.095 / 15.014.099 / 15.024.095 / 15.01

Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks c4a67979, 2026-11-11; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6f0a70e1, 2026-11-17; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks f3875362, 2026-11-11.

Parity

CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0014 and p99 <= 0.0067 (absolute error of the output flow).

variantkernels error vs original, mean / p99GPU, torchkernels used
neuflow-v20.00075 / 0.00425RTX 5060 Ti, torch 2.14.1+cu130raft_flow@a101a33607b6:sm_120
neuflow-v2-sintel0.00084 / 0.0046RTX 5060 Ti, torch 2.14.1+cu130raft_flow@a101a33607b6:sm_120
neuflow-v2-things0.00094 / 0.00429RTX 5060 Ti, torch 2.14.1+cu130raft_flow@a101a33607b6:sm_120

Limitations

  • Portable at 375 x 1242 runs 15.1 ms vs 12.2 ms for the authors' fp16 lookup (24 % slower, launch-bound; at 1080 x 1920 it is faster, 71.4 vs 75.8 ms): lookup, matching and upsampling stay float32 on purpose (lower KITTI EPE); fp16 lookup or channels_last did not help.
  • Kernels (raft_flow) run 12.0 ms / 205 MB, faster than the original, but the binary is not on Hugging Face yet; it is built locally and found through FASTFORKS_KERNEL_PATH.

Example output

One fixed input through this checkpoint, the same for every model of the task (portable · float16 · NVIDIA GeForce RTX 5060 Ti).

Frame 1
Frame 2
Output
Motorcycle, Middlebury 2014 stereo (left and right view, downscaled 4x), frame 1
Frame 1 · 100%

Input: Motorcycle, Middlebury 2014 stereo (left and right view, downscaled 4x), Scharstein et al., Middlebury College (research dataset, no licence stated).