Models · Optical flow

SEA-RAFT

SEA-RAFT (Princeton, 2024): a faster and more accurate successor of RAFT for dense optical flow, needing only 4 refinement steps. BSD-3-Clause code and weights; some training sets are research-only.

stable Fork of princeton-vl/SEA-RAFT@9137517 · backends: portable, kernels

Checkpoint

Paper 2024 Added in 0.2.0 Evaluated with 0.2.0

Kernels, RTX 5060 Ti 16 GB 73.4 ms 1080x1920, fp16
vs the faster original 5.0× original fp16: 367 ms
Peak memory 425 MiB original: 7,385 MiB
Accuracy vs original fp32 +0.000 kernels EPE, KITTI 2015 train (0.880 original)

Use it

pip install fastforks

Lean install with only what this model needs: pip install fastforks-core

from fastforks import flow

model = flow.load("sea-raft-m")  # kernels on a supported GPU, else portable PyTorch
result = model(frame1, frame2)
result.flow
result.backend  # what ran: mode, device, dtype, kernels
fastforks bench sea-raft-m

Times the model on this machine. load(…, mode="portable") or mode="kernels" picks a backend; usage guide.

Speed and memory

Every backend on one GPU (RTX 5060 Ti 16 GB) and the same input, the authors’ code included. Kernels runs fp16. Portable runs fp16 on CUDA.

BackenddtypeTime per call (ms)vs original fp16Peak memory
allocated / reserved
Load (s)First call (s)
Original fp32upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0fp32 5040.73×8,503 MiB/8,580 MiB1.20.51
Original fp16torch.autocast(fp16) around the call; upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0fp16 367–7,385 MiB/8,570 MiB0.50.37
Portable fp32fp32not measured ––––
Portablefp16 2701.4×6,060 MiB/6,180 MiB0.30.27
Kernelsfp16 73.45.0×425 MiB/454 MiB0.40.07
Portable + torch.compile 1911.9×5,869 MiB/5,920 MiB34 (build)–
TensorRT fp16fp16 1322.8×10.6 GiBenginenot comparable70 (build)–

Measured on RTX 5060 Ti 16 GB, torch 2.14.1+cu130, 2026-11-10 to 2026-11-16. First call, 3 warm-up, peak reset, median of 3 rounds x median of 5. Ratios divide two times of this table (same GPU, same input). Original fp16 is the authors’ own precision option, else torch.autocast. torch.compile and TensorRT: build time instead of load time. TensorRT memory (engine, not comparable) is the GPU memory the engine took, read from the driver because TensorRT allocates outside PyTorch, so it can't be compared with the PyTorch peaks of the other rows.

Other devices

DeviceBackenddtypems / callPeak memoryOutput vs referenceInput
Apple M1 Pro 16 GB MPS, torch 2.14.1, 2026-11-14Originalfp322637 12.2 GiB process–1080x1920
Portablefp161635 9,732 MiB processmatches1080x1920
Portablefp321820 12.2 GiB processmatches1080x1920
Kernelsfp161220 4,752 MiB processmatches1080x1920

CUDA memory is torch’s peak allocated / reserved; Apple has no such counter, so the Mac rows show the process footprint (weights, Python and torch included). “Output vs reference” compares the device’s output with the stored golden output under the card’s tolerance; – = not checked on that device. Kernels “not implemented” = no kernels for this model on that backend yet: it runs portable there.

Licence and weights

Commercial use allowed

Code
BSD-3-Clause LICENSE
Weights
BSD-3-Clause source
Training data

Our reading of the licences, not legal advice.

Weights

Tartan-C-T-TSKH432x960-M.safetensors 78.8 MB· hosted on Hugging Face

fastforks downloads and checks these files on first use (fastforks download sea-raft-m).

Accuracy

The test set through the authors’ code and through ours: same checkpoint, same protocol. The distance between the original’s fp32 and fp16 runs is what half precision alone costs; our runs should stay within it.

KITTI 2015 train

EPE / Fl-all %trained on this dataset’s public images: not zero-shot

RundtypeEPE ↓Fl-all % ↓Δ EPE vs fp32 originalpairsms / pair
Original fp32fp320.8802.51 reference 200 86.8
Original fp16fp160.8802.51 +0.000 200 50.7
Portable fp32fp320.8802.51 ±0.000 200 78.9
Portablefp160.8802.51 +0.000 200 46.3
Kernelsfp160.8802.50 +0.000 200 20.2

Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the authors’ code, same checkpoint and protocol. Measured on original fp32, original fp16: RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-11; portable fp32: RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6f0a70e1, 2026-11-17; portable, kernels: RTX 5060 Ti, torch 2.14.1+cu130, fastforks f2992269+unc, 2026-11-10.

Closeness to the original

Beyond the test-set scores: the raw output of ours against the original’s fp32 output on the same input, pixel by pixel. CPU, float32: max |diff| <= 0.00025 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.002 and p99 <= 0.014 (absolute error of the output flow).

CheckToleranceMeasured
CPU fp32, max |diff|≤ 0.00025 not recorded
GPU default dtypes, mean |diff|≤ 0.002 0.00103
GPU default dtypes, p99 |diff|≤ 0.014 0.00459
gpu_mean_kernels≤ 0.0041 not recorded
gpu_p99_kernels≤ 0.024 not recorded

Recorded for sea-raft-m on RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-12, with these kernels: cudnn_hacccutlass_convraft_flow

Details

SEA-RAFT (Wang, Lipson & Deng 2024): RAFT with ResNet encoders, an initial flow regressed from the context features and a ConvNeXt update block, so 4 iterations suffice; correlation pyramid at 1/8 resolution, convex upsampling. The authors' released weights (pretrained on TartanAir); the network is ported from princeton-vl/SEA-RAFT. `sea-raft-m` / `sea-raft-s` are the general checkpoints (trained on the FlyingThings3D, Sintel, KITTI and HD1K mix); the `-spring` variants are made for 1080p footage and work at half resolution, as upstream evaluates them: the pair is halved (bilinear), the flow is upsampled and doubled. Paper (zero-shot after FlyingChairs + FlyingThings, the C-T weights): KITTI 2015 train EPE 4.29 / Fl-all 14.2 % (M), Sintel train EPE 1.21 clean / 4.04 final (M); M with 12 iterations (`iters=12`) is the paper's SEA-RAFT (L).

Input / output
Input: two RGB images (consecutive frames). Output: per-pixel motion from the first to the second (dx, dy in pixels).
Input size
any H x W >= 128, padded to a multiple of 8; tested up to 1024x2048 (8,509 MiB allocated)
Options
  • iters=4>= 1· changes the output
Outputs
flow
CUDA kernels
  • raft_flowRAFT optical flow kernels
  • cutlass_convCUTLASS convolutions with the recurrent update (GRU) fused in
  • cudnn_haccconvolutions with fp16 accumulation (cuDNN)
Apple (MPS)
flow_metalMetal: RAFT correlation lookup without a volume. fp32, fp16
Full reference, generated from the repository

SEA-RAFT (Princeton, 2024): a faster and more accurate successor of RAFT for dense optical flow, needing only 4 refinement steps. BSD-3-Clause code and weights; some training sets are research-only.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
sea-raft-msynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
504
367
270
73.4
191
132
8503 / 8580
7385 / 8570
6060 / 6180
425 / 454
5869 / 5920
11110 / 11130
sea-raft-m-kittisynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
504
367
270
73.3
191
132
8503 / 8580
7385 / 8570
6060 / 6180
425 / 454
5869 / 5920
11110 / 11130
sea-raft-m-springsynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
96.1
56.7
46.8
23.5
34.6
4.43
759 / 858
761 / 812
652 / 700
170 / 194
624 / 640
291 / 318
sea-raft-m-thingssynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
504
367
270
73.0
192
132
8503 / 8580
7385 / 8570
6060 / 6180
425 / 454
5869 / 5920
11110 / 11130
sea-raft-ssynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
365
300
211
50.6
150
108
8462 / 8554
7323 / 8502
6019 / 6136
404 / 452
5828 / 5876
11043 / 11078
sea-raft-s-kittisynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
365
299
210
50.7
150
108
8462 / 8554
7323 / 8502
6019 / 6136
404 / 432
5828 / 5876
11043 / 11078
sea-raft-s-springsynthetic (1080x1920)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
67.1
42.4
32.8
16.2
23.8
3.01
718 / 814
699 / 764
611 / 656
150 / 172
582 / 596
215 / 278
Details
  • Original code: https://github.com/princeton-vl/SEA-RAFT at 9137517ba24e
  • Code licence: BSD-3-Clause (permissive); files: LICENSE
  • Weights licence: BSD-3-Clause
  • Licence evidence: https://huggingface.co/MemorySlices/Tartan-C-T-TSKH432x960-M/blob/85a47dc9096cca6e907ef9100dc1847a19b26a2a/README.md
  • Training data: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (TartanAir pretraining, FlyingChairs, FlyingThings3D (part of SceneFlow), then a mix of FlyingThings3D, Sintel, KITTI 2015 and HD1K (TSKH))
    • sea-raft-m-kitti: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (TartanAir, FlyingChairs, FlyingThings3D, the TSKH mix, then fine-tuned on KITTI 2015)
    • sea-raft-m-spring: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark, Spring (scene flow / optical flow / stereo, University of Stuttgart) (TartanAir, FlyingChairs, FlyingThings3D, the TSKH mix, then fine-tuned on Spring)
    • sea-raft-m-things: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa) (TartanAir, FlyingChairs, then FlyingThings3D (part of SceneFlow))
    • sea-raft-s-kitti: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (TartanAir, FlyingChairs, FlyingThings3D, the TSKH mix, then fine-tuned on KITTI 2015)
    • sea-raft-s-spring: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark, Spring (scene flow / optical flow / stereo, University of Stuttgart) (TartanAir, FlyingChairs, FlyingThings3D, the TSKH mix, then fine-tuned on Spring)
  • Options: iters = 4 (>= 1)
  • Kernel libraries: raft_flow, cutlass_conv, cudnn_hacc
  • Variants: sea-raft-m, sea-raft-m-kitti, sea-raft-m-spring, sea-raft-m-things, sea-raft-s, sea-raft-s-kitti, sea-raft-s-spring
  • Outputs: flow
  • Input size:
    • sea-raft-m, sea-raft-m-kitti, sea-raft-m-things, sea-raft-s, sea-raft-s-kitti: any H x W >= 128, padded to a multiple of 8
    • sea-raft-m-spring, sea-raft-s-spring: any H x W >= 256, padded to a multiple of 8
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): sea-raft-m 1024x2048 (8.3 GiB peak allocated, 9.4 reserved); sea-raft-m-kitti 1024x2048 (8.3 GiB peak allocated, 9.4 reserved); sea-raft-m-spring 1988x2880 (4.1 GiB peak allocated, 4.8 reserved); sea-raft-m-things 1024x2048 (8.3 GiB peak allocated, 9.4 reserved); sea-raft-s 1024x2048 (8.3 GiB peak allocated, 9.3 reserved); sea-raft-s-kitti 1024x2048 (8.3 GiB peak allocated, 9.3 reserved); sea-raft-s-spring 1988x2880 (4.1 GiB peak allocated, 4.8 reserved)
  • sea-raft-m 1080x1920, first call: portable + torch.compile 34 s (cold Inductor cache), TensorRT fp16 70 s (ONNX + engine).
  • sea-raft-m-kitti 1080x1920, first call: portable + torch.compile 35 s (cold Inductor cache), TensorRT fp16 70 s (ONNX + engine).
  • sea-raft-m-spring 1080x1920, first call: portable + torch.compile 23 s (cold Inductor cache), TensorRT fp16 47 s (ONNX + engine).
  • sea-raft-m-things 1080x1920, first call: portable + torch.compile 35 s (cold Inductor cache), TensorRT fp16 74 s (ONNX + engine).
  • sea-raft-s 1080x1920, first call: portable + torch.compile 35 s (cold Inductor cache), TensorRT fp16 68 s (ONNX + engine).
  • sea-raft-s-kitti 1080x1920, first call: portable + torch.compile 32 s (cold Inductor cache), TensorRT fp16 67 s (ONNX + engine).
  • sea-raft-s-spring 1080x1920, first call: portable + torch.compile 22 s (cold Inductor cache), TensorRT fp16 39 s (ONNX + engine).
  • sea-raft-m original (fp32) 1080x1920: upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-m original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-m-kitti original (fp32) 1080x1920: upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-m-kitti original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-m-spring original (fp32) 1080x1920: upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=-1
  • sea-raft-m-spring original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=-1
  • sea-raft-m-things original (fp32) 1080x1920: upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-m-things original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-M.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-s original (fp32) 1080x1920: upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-s original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-s-kitti original (fp32) 1080x1920: upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-s-kitti original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=0
  • sea-raft-s-spring original (fp32) 1080x1920: upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=-1
  • sea-raft-s-spring original 1080x1920: torch.autocast(fp16) around the call; upstream RAFT (spring-S.json architecture) + evaluate.py calc_flow, iters=4, scale=-1
  • sea-raft-m load / first call: original 0.53 s / 0.367 s, portable 0.29 s / 0.27 s, kernels 1.15 s / 0.072 s
  • sea-raft-m-kitti load / first call: original 0.67 s / 0.367 s, portable 1.02 s / 0.27 s, kernels 1.14 s / 0.072 s
  • sea-raft-m-spring load / first call: original 0.57 s / 0.06 s, portable 0.74 s / 0.052 s, kernels 1.14 s / 0.023 s
  • sea-raft-m-things load / first call: original 0.55 s / 0.367 s, portable 0.29 s / 0.27 s, kernels 1.13 s / 0.072 s
  • sea-raft-s load / first call: original 0.3 s / 0.3 s, portable 0.18 s / 0.211 s, kernels 1.1 s / 0.05 s
  • sea-raft-s-kitti load / first call: original 0.3 s / 0.299 s, portable 0.27 s / 0.211 s, kernels 1.1 s / 0.05 s
  • sea-raft-s-spring load / first call: original 0.31 s / 0.043 s, portable 0.34 s / 0.039 s, kernels 1.11 s / 0.016 s
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5

Measured

Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portableportable fp32kernelsnote
sea-raft-mKITTI 2015 train, EPE / Fl-all %0.880 / 2.510.880 / 2.510.880 / 2.510.880 / 2.510.880 / 2.50trained on this dataset's public images; not a zero-shot result
sea-raft-m-kittiKITTI 2015 train, EPE / Fl-all %0.808 / 2.160.808 / 2.160.808 / 2.160.808 / 2.160.808 / 2.16trained on this dataset's public images; not a zero-shot result
sea-raft-m-springKITTI 2015 train, EPE / Fl-all %6.779 / 23.756.775 / 23.746.775 / 23.746.779 / 23.756.783 / 23.74trained on this dataset's public images; not a zero-shot result
sea-raft-m-thingsKITTI 2015 train, EPE / Fl-all %4.293 / 14.314.290 / 14.324.289 / 14.324.295 / 14.314.307 / 14.32
sea-raft-sKITTI 2015 train, EPE / Fl-all %1.054 / 3.131.054 / 3.131.054 / 3.131.054 / 3.131.054 / 3.14trained on this dataset's public images; not a zero-shot result
sea-raft-s-kittiKITTI 2015 train, EPE / Fl-all %0.936 / 2.660.936 / 2.660.936 / 2.660.935 / 2.660.935 / 2.65trained on this dataset's public images; not a zero-shot result
sea-raft-s-springKITTI 2015 train, EPE / Fl-all %7.758 / 26.527.758 / 26.527.758 / 26.527.758 / 26.527.757 / 26.51trained on this dataset's public images; not a zero-shot result

Measured on portable: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks f2992269+uncommitted, 2026-11-10; portable32: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 6f0a70e1, 2026-11-17; kernels: NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks f2992269+uncommitted, 2026-11-10.

Parity

CPU, float32: max |diff| <= 0.00025 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.002 and p99 <= 0.014 (absolute error of the output flow).

variantkernels error vs original, mean / p99GPU, torchkernels used
sea-raft-m0.00103 / 0.00459RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-m-kitti0.00085 / 0.00411RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-m-spring0.00149 / 0.00741RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-m-things0.00083 / 0.00347RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-s0.00149 / 0.00788RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-s-kitti0.00141 / 0.00847RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120
sea-raft-s-spring0.0027 / 0.01574RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, raft_flow@a101a33607b6:sm_120

Example output

One fixed input through this checkpoint, the same for every model of the task (portable · float16 · NVIDIA GeForce RTX 5060 Ti).

Frame 1
Frame 2
Output
Motorcycle, Middlebury 2014 stereo (left and right view, downscaled 4x), frame 1
Frame 1 · 100%

Input: Motorcycle, Middlebury 2014 stereo (left and right view, downscaled 4x), Scharstein et al., Middlebury College (research dataset, no licence stated).