Every Output Should Know Where It Came From

The problem — Outputs travel. A segmentation mask gets emailed, a profile lands in a slide, a count matrix is copied into a shared drive — and at that moment it is detached from the run that made it. Verification asks whether the whole pipeline re-runs to the same result; provenance asks a narrower, harder question: if this one artifact is all you have, can you still say which image, which model version, which parameters, and which QC verdict produced it? When the answer is no, the output is an orphan — trusted on appearance, impossible to audit, dangerous to combine with anything else.

What it is / how it works — Provenance makes lineage a first-class property of the output, written alongside the data rather than left in a logbook. Each artifact carries: the input identity (ideally a content hash, so the source image is named by its bytes, not a mutable path), the provenance metadata — instrument, acquisition settings, processing steps — that the microscopy metadata perspective separates from quality metadata, the pinned code/model versions and parameters, and the QC decision attached at the report step. Standards make this portable: OME-NGFF carries acquisition metadata in the array, and tools like Micro-Meta App capture instrument and settings against community specs. The discipline echoes Sculley et al. — undeclared consumers are the hazard, and an output with no lineage is a dependency nobody can trace.

Where it breaks — The general failure: lineage lives in the pipeline's head, not on the artifact, so the moment an output leaves the run it becomes unaccountable — and this bites across every platform. A Cell Painting profile pooled into a reference map with no record of its plate, illumination-correction version, or normalization silently injects a batch effect. A digital pathology slide-level score reported without the scanner, stain lot, and model version cannot be reconciled when a second cohort disagrees. A light-sheet segmentation reused months later, with no note of the deconvolution library that shaped it, is irreproducible structure masquerading as biology. A spatial omics count matrix detached from its registration parameters cannot be re-derived if the assignment is questioned. The fix is the same everywhere: emit lineage with the output, so the artifact survives separation from its run — the precondition for reanalysis.

References

• Huisman et al. (2019). A Perspective on Microscopy Metadata: Data Provenance and Quality Control.
• Sculley et al. (2015). Hidden Technical Debt in Machine Learning Systems. NeurIPS.
• Rigano et al. (2021). Micro-Meta App: an interactive tool for collecting microscopy metadata based on community specifications. Nature Methods.