Corvus ISR Publishes Transparent Benchmark Showing Tracker Improvements

Corvus ISR tracker benchmark matrix (seed 1337) — The published matrix — every row reproducible. Source: corvusisr.com/benchmark

Corvus ISR’s latest public tracker benchmark offers a transparent look into the performance of its two WAMI exploitation models, emphasizing the importance of reproducibility and disciplined testing in software engineering. Using a fixed-seed synthetic scene with perfect ground truth, the benchmark provides byte-identical comparisons that eliminate variables, ensuring fair evaluation of different tracker versions.

The two models compared are the v1 “greedy nearest-neighbour,” a simple baseline with two-pass greedy association, and the more advanced v2 “confirmed-track auction,” featuring a three-tier auction association, velocity-consistency gating, and confidence-decayed coasting. The benchmark results show significant improvements: ID switches per minute dropped from 2,042 to 1,183 (a 42.1% reduction) in the baseline configuration with 150 movers running at 2fps. Similar gains are observed with higher density and degraded conditions, demonstrating the robustness of v2 under stress.

One notable aspect of this release is the strictness of the ID switch metric, which counts every change in track identity—more rigorous than standard MOT challenges. This honesty in measurement underscores that even the latest tracker still commits thousands of errors per minute, emphasizing the ongoing challenges in the field. The purpose of publishing these numbers is to foster measurement-based development, encouraging vendors to aim for accountability rather than marketing-only success stories.

From an engineering perspective, v2 maintains real-time performance, averaging around 1.2ms per sensor tick at dense scenes of 400 movers, with worst-case timing at 5ms against a 10ms budget. The entire process is transparent and reproducible: anyone can run the benchmark themselves by visiting the live demo and pressing “Run benchmark” — no signup or NDA required. This demonstrates a disciplined approach to test automation and validation that aligns with best practices in QA and software development.

Central to this effort is the automated AI executor that built v2 against a written acceptance contract, which was then independently reviewed before deployment. The synthetic nature of the scene, where every pixel is generated and no real-world entities are involved, ensures controlled testing conditions. Corvus ISR’s approach exemplifies how rigorous, reproducible benchmarks can drive meaningful progress, even in complex perception tasks.

Readers interested in the details can explore the public benchmark to see the full results. This commitment to transparency and measurement invites everyone to reproduce it live and verify the improvements firsthand. Such disciplined test design sets a standard for engineering teams striving for robust, honest evaluation of their perception systems.