The problem — After segmentation, feature extraction, and batch correction, a pipeline produces a matrix of perturbation profiles that always looks reasonable on its face. Without an explicit reproducibility gate, a failed assay and a working one are indistinguishable — both emit numbers. Downstream biologists receive hits nominated from noise.
What it is / how it works — Percent replicating asks one question: are replicate treatments more similar to each other than to randomly sampled non-replicates? For each perturbation, the median pairwise Spearman (or cosine) correlation of its replicates is compared against a permutation null that controls for dose, plate position, and replicate cardinality. A perturbation "replicates" when its correlation exceeds the 95th percentile of that null; percent replicating is the fraction of all perturbations that pass. The metric was formalised by Way et al. 2022 and unified with mAP-based retrieval in the copairs framework (Kalinin et al. 2025). In practice, copairs handles the pair construction, null sampling, and thresholding — it is the standard implementation in the cytomining ecosystem.
Where it breaks — The metric is a lagging indicator: it reflects the accumulated quality of every upstream stage. A low score is a pipeline diagnosis signal, not a biological conclusion. The most common root causes are upstream — failed illumination correction introduces systematic well-position signal that swamps biology, and residual batch effects after batch correction inflate cross-plate distance for the wrong reasons. Tuning the metric threshold to compensate is the wrong response; fix the upstream stage.
The Fovea position: percent replicating is a gate, not an optimisation target. Treat a score below ~40–50% as a stop condition and trace the failure back through correction, features, and segmentation before reporting anything. For context on why retrieval-based metrics matter here, see Measure Where It Matters and metrics that do not match domain interest.