Method / tool

copairs

Python library for computing mean Average Precision (mAP) and % Replicating between perturbation profiles in image-based profiling; the standard readout tool for quantifying phenotypic activity and assay quality in Cell Painting workflows.

The problem — After segmentation, feature extraction, and batch correction, you have a matrix of perturbation profiles. The question every assay must answer before a result reaches a biologist: are replicate treatments actually more similar to each other than to random pairs drawn from the same plate? Without a principled retrieval metric, a broken assay looks identical to a working one — both produce numbers, neither produces signal.

What it is / how it works — copairs is the cytomining ecosystem's standard tool for pairwise similarity retrieval. Given a profile matrix and a metadata table, it identifies all positive pairs (replicates of the same perturbation) and negative pairs (non-matching perturbations), computes cosine similarity, and ranks all negative pairs against each positive. From that ranking it derives mean Average Precision (mAP) — the area under the precision-recall curve averaged across all queries. A permutation null over shuffled labels converts raw mAP into a corrected score, and Percent Replicating is the fraction of perturbations whose mAP exceeds the 95th percentile of that null.

The library handles the combinatorial bookkeeping — positive/negative pair construction, null sampling, and statistical thresholding — that is otherwise error-prone to implement by hand. It was formalised in Kalinin et al. 2025 (Nature Communications), which unified mAP-based retrieval with earlier ad-hoc replicate metrics under one information-retrieval framework validated on CPJUMP1 and related datasets.

Where it fits in the pipeline — copairs lives at the readout stage, after sphering and Harmony correction. Running it earlier — on raw or incompletely corrected profiles — conflates batch signal with biological similarity, inflating mAP for the wrong reasons and hiding genuine failures.

Where it breaks — mAP is sensitive to the null distribution construction. Too few negative pairs (sparse plates, unusual perturbation density) produce noisy percentile estimates. The metric also assumes profiles are already on a common scale; if upstream normalisation is inconsistent across batches, the ranking conflates batch proximity with phenotypic proximity. copairs does not diagnose why mAP is low — it only tells you it is. Root-causing a low % Replicating still requires inspection of the illumination correction, segmentation quality, and batch correction steps upstream.

A high mAP for positive controls does not validate an entire assay. Controls and test perturbations can occupy different regions of feature space; confirm that test-compound mAP distributions are also sensible before reporting replicate concordance as a quality pass.

References

Appears in these notes

  • Cell Painting Pipelines — From Images to Phenotypic ProfilesA Cell Painting profile is the output of a long pipeline — illumination correction, segmentation, feature extraction, batch correction, then a reproducibility readout — and the number that matters (percent-replicating / mAP) lives only at the end. Every stage upstream can silently degrade it.
  • Percent ReplicatingThe reproducibility metric that catches a broken assay before it reaches a biologist.

← Back to the constellation