The problem — A profiling pipeline can run end-to-end and produce feature matrices without ever producing signal. If you skip a principled retrieval evaluation, broken assays are indistinguishable from working ones: both return numbers, only one returns biology. The failure is invisible until a biologist acts on a false hit or a screen delivers uniform noise.
What it is / how it works — Retrieval formalises the question "does my assay work?" as an information-retrieval task. For each query perturbation, compute cosine similarity to every other profile in the embedding space, rank the list, and ask: do the known-positive matches (biological replicates, same gene, same mechanism-of-action class) rank above a permutation null? Two complementary metrics quantify this. Percent Replicating is the fraction of perturbations whose replicate similarity exceeds the 95th percentile of a null drawn from non-matching pairs — a binary gate per perturbation. Mean Average Precision (mAP), formalised by Kalinin et al. 2025 in the copairs framework, integrates the full ranking curve into a single score and supports a p-value against the same null, making it suitable for comparing assay variants or profiling modalities at scale. CPJUMP1 (Chandrasekaran et al. 2024) established the public benchmark at which both metrics are evaluated across 136,000 chemical and genetic perturbations.
Where it breaks — Retrieval is a system-level test, not a module-level one. A low mAP score is a diagnosis signal, not a conclusion: it could trace to illumination-correction failure, over-aggressive batch correction erasing biology, poor feature selection, or a genuinely uninformative perturbation. CPJUMP1 cross-modality compound–gene matching sits barely above chance, which is a sobering baseline — not a flaw in the metric, but a reflection of how much headroom the upstream pipeline must earn. Retrieval also assumes the positive-pair definition is scientifically valid; if "same MoA" annotations are noisy or incomplete, the null comparison is undefined and mAP loses its interpretive grounding.