Method / tool

mAP

Mean Average Precision (mAP) summarizes object detection performance by averaging the area under the Precision-Recall curve across object classes and IoU thresholds — the standard leaderboard metric for COCO-style detection benchmarks, but one that aggregates over deployment-irrelevant confidence thresholds and masks class-specific failure modes.

The problem — Object detection models in microscopy are routinely compared by mAP, borrowed wholesale from computer-vision benchmarks. The number looks authoritative: a single scalar, higher is better, computed on a standard dataset. But mAP averages over confidence thresholds no analyst will ever deploy at, collapses class-specific failure modes into one mean, and rewards recovery at arbitrarily low precision points that correspond to no real operating point. A model that achieves high mAP can perform worse than a simpler baseline at every threshold a biologist would actually use.

What it is / how it works — Average Precision (AP) for a single class is the area under the Precision-Recall curve, sampled at all confidence thresholds; Mean AP averages AP across classes. COCO's mAP@[0.50:0.05:0.95] further averages AP at ten IoU thresholds from 0.5 to 0.95, rewarding models that produce tight bounding boxes or accurate masks across a range of overlap criteria. This is stricter than older Pascal VOC mAP@0.5 and was introduced in the COCO 2014 paper by Lin et al.. For instance segmentation, mAP is computed on mask IoU rather than box IoU, making it directly comparable to mask-quality metrics like Panoptic Quality.

In microscopy, mAP is most commonly encountered when evaluating nucleus or cell detectors on annotated benchmark datasets — for example, as a supplementary metric alongside SEG in cell-tracking evaluations or alongside F1 in pathology detection challenges.

Where it breaksMetrics Reloaded identifies several failure modes relevant to bioimage detection: mAP weights all confidence thresholds equally, but practitioners operate at a single threshold chosen by the downstream analysis. mAP over-summarizes class heterogeneity — a strong mean can hide that rare cell types (e.g., mitotic figures, multinucleated cells) are consistently missed. And because mAP is a relative ranking metric, a model that wins a benchmark can still have unacceptable absolute recall on the rare events that matter most to the biology.

The Fovea operating stance: report per-class AP at the operating threshold you will actually deploy, not a benchmark mean. Follow the principle of problem-aware metric selection — choose the metric from the decision, not from what the benchmark infrastructure already computes.

References

Appears in these notes

  • Why mAP, Dice, and SSIM Are Not EnoughDice, IoU, mAP, and SSIM measure overlap or similarity against a reference — not whether the result supports the decision the experiment exists to make. The discipline is problem-aware metric selection: derive the metric from the question, not from what is easy to compute.

← Back to the constellation