The frame-age problem
A perception pipeline that looks healthy can be acting on a frame that is 400 ms old. Freshness is a feature you have to build.
When a camera feeds a queue, the consumer reads the oldest frame in the queue. Under load that frame can be hundreds of milliseconds stale — and nothing in the graph looks broken. The detector runs, the boxes appear, the counters tick up. The vehicle is just steering by the past.
We measured it: a naive queue served frames 396 ms stale on average. A mailbox that keeps only the newest frame and drops the rest served them 16.9 ms stale. Same camera, same detector — the difference is entirely in which frame you choose to believe.
Freshness over throughput
For a moving vehicle, an old correct answer is a wrong answer. Throughput is a benchmark number; freshness is what keeps the control loop honest. The mailbox throws away work on purpose, and that is the right trade.
The speed of light sucks. — Carmack
The lesson generalises past cameras: any time a slow stage sits behind a fast one, ask what happens to the backlog. If the answer is “it queues,” you have built a time machine pointed the wrong way.