The problem — Most cell trackers commit to a single segmentation mask before solving the linking problem. In dense, 3D+t light-sheet datasets — a zebrafish embryo, a nematode at high cell density — the correct segmentation is genuinely ambiguous: cells touch, partially occlude, and divide in rapid succession. Committing to one mask propagates errors into linking; the tracker then has bad inputs for every subsequent frame.
What it is / how it works — ultrack (Bragantini et al., Nature Methods 2025) decouples segmentation from tracking by generating a graph of segmentation hypotheses — multiple overlapping candidate masks per time point — and then solving a global integer-programming optimisation that selects the subset of candidates most consistent across time. The solver jointly optimises detection confidence, shape consistency, and temporal continuity, effectively choosing the best segmentation in the context of the whole trajectory rather than frame by frame. The package is pure Python, pip-installable, and supports OME-Zarr/NGFF inputs natively, making it a natural downstream consumer of PetaKit5D output. It ships Fiji, napari, and command-line interfaces and has been validated on terabyte-scale zebrafish, Drosophila, and C. elegans embryo time-lapses, with state-of-the-art performance on the Cell Tracking Challenge.
Where it fits in the Fovea workflow — ultrack is the preferred tracking engine in light-sheet pipelines when the acquisition is dense or the segmentation quality is uneven — conditions that defeat TrackMate's LAP linker. It sits at the segment/features stage, consuming fused, registered volumes and producing lineage-resolved track tables. Its hypothesis-graph formulation makes it significantly more robust to the exact failure modes the pipeline note describes: deep, low-SNR regions with no reliable single mask.
Where it breaks — The integer-programming solver is computationally heavier than LAP linking; terabyte-scale runs require HPC infrastructure and careful memory budgeting. Generating too many segmentation hypotheses (overly permissive detectors) can make the graph intractable. And like any tracker, ultrack's output is only as good as the signal: when cells are sub-resolution or SNR is below the floor where any candidate mask is reliable, tracking fails before the optimiser is even invoked.