Verification and Validation Are Two Different Questions

The problem — "Is the pipeline working?" hides two distinct questions that fail independently. Verification asks whether the system was built right: does it compute what it is specified to compute, deterministically, and can the same inputs reproduce the same outputs tomorrow? Validation asks whether it is the right system: do its outputs correspond to biological reality and answer the actual question? A pipeline can be fully verified and completely invalid — perfectly reproducible nonsense — or validated once on a benchmark yet non-reproducible in practice. Teams routinely report one and imply the other, which is precisely how confidently wrong results reach a slide deck.

What it is / how it works — Keep the two axes explicit, because each has its own failure surface and its own evidence:

• Verification — build it right. The pipeline is deterministic, version-pinned (code, environment, model weights, parameters), and re-executable from a provenance record. The enemy is system-level entanglement — Sculley's CACE principle, changing anything changes everything, where data dependencies are more dangerous than code. Evidence is a green regression suite and a run you can reproduce, not a published figure. → Verification: a pipeline you can re-run.
• Validation — build the right thing. The output is fit for purpose against the biological question, judged with metrics that actually reflect the domain interest rather than whichever score is convenient (Metrics Reloaded). In microscopy the obstacle is structural: the ground truth is scarce, synthetic, or absent. → Validation without a ground truth.

The order matters: validation without verification is unrepeatable luck; verification without validation is rigor in service of the wrong answer. You need both, and you need to know which one you are claiming.

Where it breaks — Conflation is the failure mode, and it looks different per platform. A Cell Painting profiler can be bit-for-bit reproducible (verified) while its features fail to replicate across plates (invalid). A digital pathology model can validate on one cohort yet silently diverge when a scanner or stain lot changes — a verification/provenance gap masquerading as a validation success. A light-sheet tracking pipeline can pass on a curated clip but degrade with imaging interval and colony size, so a single validation number is itself unverified. And across all of them, evaluation noise and misaligned metrics let benchmark gains outrun real utility. The discipline is to treat V&V as two ledgers — reproducibility evidence and fitness-for-purpose evidence — and never let a checkmark in one column be read as a checkmark in the other. This is the backbone of verified outputs and the reason a model isn't the system.

References

• Verification and Validation for Trustworthy Scientific Machine Learning (2025).
• Sculley et al. (2015). Hidden Technical Debt in Machine Learning Systems. NeurIPS.
• Maier-Hein, Reinke et al. (2024). Metrics Reloaded: Recommendations for Image Analysis Validation. Nature Methods.
• Varoquaux & Cheplygina (2022). Machine Learning for Medical Imaging: Methodological Failures and Recommendations. npj Digital Medicine.