The 71 transposes that were one operation
How a feature matcher that refused to compile for a neural accelerator turned out to be a single idea wearing 71 masks — and why proving the fix was exact mattered more than the 23× speedup.
XFeat is a state-of-the-art lightweight feature matcher. The Hailo-8 is a neural accelerator that does 26 TOPS and nothing else well. Getting the first onto the second should have been a compile step. Instead the compiler died: UnsupportedShuffleLayerError on 71 Transposes, 72 Slices and 70 Unsqueezes, all five-dimensional.
The temptation is to fight all 213 operations. The breakthrough was seeing that they are not 213 problems, or even 71 — they are one. They are _unfold2d(x, ws=8), written with Tensor.unfold, which torch traces as a pile of 5-D slicing. That operation has a name: space-to-depth. torch already has it as pixel_unshuffle, and the chip supports it natively. One substitution, and the graph parses.
The proof, not the claim
Here is the part I care about. A graph that compiles but computes something slightly different is the worst kind of bug: it will surface months later as a worse match rate and get blamed on quantisation, and you will never find it. So the export tool refuses to write the model until it has run torch.equal on random input and checked the whole network against the stock graph — max absolute difference 0.000e+00 across all three heads at both resolutions. The substitution is not an approximation. The tool proves it, or it writes nothing.
A HEF that compiles is not a result.
The honest part
Then the number: 701 FPS on the chip, 23× the Pi’s CPU — a figure that appears to be published nowhere else. And immediately, the caveat that makes it true: only the backbone moved. Keypoint decoding, NMS and the matcher stay in numpy, and the match is the expensive half. So the real win is not the speed. It is that the bottleneck moved from the detector to the matcher, which reframes the next optimisation entirely. A speedup you cannot spend is not a speedup; knowing exactly where the new wall is, is.
The settings that shipped came from the same discipline. On real murky footage, top_k=1024 beats 2048 — because the similarity matrix is O(N²) in keypoints, and at a fixed budget four references at 1024 beat one at 2048. Bank breadth beats keypoint density at equal cost. Chosen, not inherited.