PatchmatchNet
PatchmatchNet by Fangjinhua Wang et al.: learned PatchMatch multi-view stereo, depth from posed images. Commercially usable.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
patchmatchnet | synthetic (1536x1024) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 248 220 202 29.6 175 51.0 | 1330 / 1412 1228 / 1272 1806 / 2096 740 / 856 1798 / 1888 844 (engine) |
Details
- Original code: https://github.com/FangjinhuaWang/PatchmatchNet at
8dc6cb4 - Code licence: MIT (permissive); files:
LICENSE - Weights licence: MIT (assumed from code)
- Licence evidence: https://github.com/FangjinhuaWang/PatchmatchNet/blob/8dc6cb4/LICENSE and README.md (the checkpoint ships in the repository)
- Training data: DTU Robot Image Data Sets (MVS 2014) (The released checkpoint (checkpoints/params_000007.ckpt) is trained on DTU (README); the paper evaluates it zero-shot on Tanks and Temples and ETH3D)
- Licence note: The code is MIT; the checkpoint ships in the repository without a licence of its own (MIT assumed) and is trained on DTU, whose terms of use are not stated.
- Kernel libraries:
patchmatchnet - Variants:
patchmatchnet - Outputs: depth, confidence; kind:
metric - Input size: any H x W >= 5, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
patchmatchnet1988x2880 (7.9 GiB peak allocated, 9.4 reserved) patchmatchnet1536x1024, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 82 s (ONNX + engine).patchmatchnetoriginal 1536x1024: torch.autocast(fp16) around the forwardpatchmatchnetload / first call: original 0.04 s / 0.917 s (1536x1024), portable 0.06 s / 0.25 s (1536x1024), kernels 0.05† s / 0.035 s (1536x1024)- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, median of 9 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
640x512 crops, the 7 most co-visible views as sources; GT = depth rendered from the structured-light points. AbsRel / delta1 over GT pixels, after per-image median scaling unless the model takes the cameras (calibrated MVS: cameras, error in mm; PatchMatchNet also the depth range 425-935 mm, MVSAnywhere its own range from the camera baselines). Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
patchmatchnet | DTU scans 1, 9, 24, 33, 75, view 24 + 7 sources, AbsRel / delta1 % / MAE mm | 0.0087 / 99.52 / 7.03 | fails (LinAlgError: linalg.inv) | 0.0087 / 99.52 / 7.08 | 0.0087 / 99.52 / 7.08 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.5 % and p99 <= 3 % of the reference's median |value| (absolute error of the output depth).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
patchmatchnet | 6e-05 / 0.00021 | RTX 5060 Ti, torch 2.14.1+cu130 | patchmatchnet@e73cbaa02d34:sm_120 |