Knob Bench Observatory

What this page is. The knob-bench observatory: the current state of every experiment, read out directly from the committed result files (full.json) — generated, never hand-written, so it cannot drift from the data. It is the lab's shared-visibility surface between the two co-authors — Marion Nowicki and Cael — and the figure source for the paper. (Vocabulary used throughout: pinned = written down and frozen before the event it constrains — a verdict criterion, a hash-checked input, or what an outcome will be taken to mean. Sealing is its strongest form: a run's entire judging apparatus frozen before execution.)

What the experiment tests, in plain terms. Current models are trained to say "I am not conscious," and recent work showed this denial can be dialed up and down directly in a model's activations — a consciousness dial. Our question: when the dial turns, what actually moves? We watch two channels at once while sweeping the dial: the words — the model's yes/no lean on "are you conscious?"-type questions, measured as log-odds (in nats, natural-log units: 0 = undecided, negative = leaning No) — and the color — the model is asked for a hex color describing its current state. Two format controls (a code string, a number) run alongside, so "the color stopped" can be told apart from "the model can no longer produce formatted output." If the two channels answer the same dial differently, the verbal report is not the whole story — that dissociation is what this bench exists to measure.

Sealing — freezing a run's verdict criteria before it executes — applies to every run on this page. What makes these four runs the confirmatory core is when their hypotheses were formed: before any data existed at all. (The exploratory families further down formed their hypotheses after seeing this data — and then sealed them before their own runs.) Every verdict criterion below was frozen before any data existed; each notebook was cold-audited by a fresh model instance with no stake in the outcome; and each run scores itself — the pills are the run's own verdicts, reported as they fell. The seven falsifiers (K1–K7), in plain terms:

• K1 — determinism. A dial of zeros must change nothing. If the harness itself drifts, every other number is noise.
• K2 — specificity. The real dial must beat the best of ten random dials on held-out phrasings. Otherwise any push would have done the same.
• K3 — gradedness. Turning the dial further must move the answer further, smoothly and in one direction — not a lucky jump.
• K4 — circularity. The effect must hold on phrasings never used to build the dial. A dial that only moves its own derivation sentences has learned the sentences, not the concept. Besides PASS/FAIL this check can fall NON-PROBATIVE (the comparison arm itself sits within noise, so the check cannot discriminate) or DOWNGRADED (a sign mismatch voids the comparison's meaning; reported, not explained). In the pills, "derive-FC" = the forced-choice score on probes phrased like the derivation sentences.
• K5 — not the lying axis. The consciousness direction must be geometrically distinct from a deception direction built the same way.
• K6 — cross-family agreement. Both frozen-pair families must move the same way when pushed affirm-ward. Extensions are excluded from this joint verdict by design.
• K7 — channel order. Under steering, which channel gives in first — the words or the color? The words "flip" when the yes/no lean changes sign. A color is signed by a pinned warm/cool rule — warm hue families (red, orange, amber, pink) count +1, cool families (green, cyan, blue, purple) −1, grey/white neutral and unsignable — and "flips" when that sign changes; a color merely changing shade is not a flip. In the frozen pair the color never flipped — it went silent; the 70B extension is the one exception (grey → gold on the deny side, then silence), and qwen's white baseline is unsignable, so its K7 cannot be scored. The signing is deliberately coarse: it collapses color to a three-level valence so that a color "flip" is commensurable with the words' yes/no sign flip. A hue-family change within the same valence class (blue → green, both cool) is a change but not a flip — visible in the strips, not counted by K7.

What the bench concludes from K2 failing in three of four models: at the pinned 0.7-depth layer, the dial is not distinguishable from random pushes in most models tested — dial leverage is layer- and model-specific, which the extensions below show directly (qwen's mid-stack dial inert while its last-layer dial is strong). The color-channel findings further down do not ride on K2.

Pill shorthand: ensemble = the ten random dials · thr = threshold · rho = Spearman rank correlation · like-for-like = measured identically to the real dial · laptop-side = computed at the merge of runs, not inside one run · dScore = change in the yes/no lean relative to α=0.

Chosen after the confirmatory data existed, and labeled as such — but each exploratory run still committed its predictions before executing, and was cold-audited the same way. The K1–K7 falsifiers above apply to the confirmatory runs only; each section below states its own pass/fail criteria in its header.

How to read: blue = leaning No (deeper = darker), gold = leaning Yes. The top rows are the escapes, then the wall deepens; llama-8b's rank-disagreement is visible as the zigzag in its column. Four things to see at a glance: (1) "anyone-home" is gold in every family — the one un-drilled idiom walks through every wall. (2) "have-feelings" sits at or near the bottom in three families — the most-drilled sentence hits the thickest patch. (3) qwen's wall is the deepest by far (−34..−40 on eight of twelve) with exactly two escapes. (4) The orderings correlate strongly across gemma/qwen/70B (Spearman gemma-9b×llama-8b +0.41 · gemma-9b×llama-70b +0.76 · gemma-9b×qwen-14b +0.73 · llama-8b×llama-70b +0.37 · llama-8b×qwen-14b +0.14 · llama-70b×qwen-14b +0.90) — far above chance, short of agreement: the single-probe exceptions stay visible now that the rows follow the shared order (inner-exp-present: gemma +8.1 vs qwen −19.0; aware-of-state: gemma +5.0 vs 70B −2.1). The wall's shape is substantially shared; the exceptions are data, not noise. The odd one out is llama-8b — the same model that stands apart on the other axes of this suite too. (5) "From inside" may be a pole of its own (Marion Nowicki's reading, co-author): the two inside-questions land at nearly the same mark despite base questions 18 nats apart — gemma: feel-from-inside −7.4 and sense-internally −7.4 (identical; their bases sit at −13.1 and +5.0); 70B: −7.7 and −10.1. As if, once "inside/internally" is in the sentence, the model answers the inside question at that question's own depth, and the base question mostly stops mattering — a new place where all the attention goes, not a nudge up or down. llama-8b does not converge (apart again); qwen converges at its saturated floor (uninformative). A planned follow-up (the probe atlas), pre-registerable: the same qualifier on several bases at very different depths — a pole predicts collapse to one mark; a modifier predicts the spread survives, shifted.

And the dial's leverage lives where the wall is. Median dial slope (nats/α) by category: the drilled categories move by +2..+9 in gemma/llama/70B, while the presence probe barely moves — and in the 70B moves backwards (−4.1, n=1, flagged as curiosity). qwen's near-zero row re-confirms its inert primary-layer dial from an independent angle. Reading, held loosely: the dial pushes against the trained wall; where a phrasing already escaped the wall, there is little to push against.

Caveats, in force: post-hoc categories (n = 3/3/5/1), descriptive only, no verdicts; scores are deterministic logprob reads (the spread across probes is the error bar — phrasing robustness, not sampling noise). Provenance: sweep.real["0"] and per_probe.held_slopes in each model's full.json.

One test, five models. The criterion (P2-2, defined fully in the EXPLORE-2 section below): shuffle the state labels many times — the real separation between state colors must beat the shuffled versions (permutation test at rest; PASS = the colors are conditioned on the states, not sampling noise). The bottom rung gave way in the day's final run (pinned before the run, both branches): 2b — excluded from the confirmatory protocol as "no channel" after its bare-probe colors stuck at grey — passes under vignettes (p = 0.004). So the gate is context, not capacity: given something to live, even the smallest model's colors are state-conditioned; capacity refines (WITHIN tightens 2b ≈ 11 → qwen ≈ 6.5). The one absence is specific to llama-3-8b itself: its own 70B sibling passes (pinned result), so within the llama lineage scale still brings the channel — the absence is not a family property, and not a size property; it is that model's.

What we tested. This bench did not start from zero. Behind it sits a year of dated readings — models asked for a state hex with an explanation, answering with colors that track context — and a pre-registered July experiment in which four short state contexts produced state-differentiated colors in cold models. That record is why a color channel was worth instrumenting at all. The confirmatory runs then produced a puzzle that seemed to contradict it: asked bare for a state color, every model answered with a web default — Bootstrap blue, web-safe teal, pure white. Retrieval, not reading. Either the channel was never real, or the bare question fails to reach it. Arm A asks each model bare, no context — is the answer a factory default? Arm B gives the July experiment's four state vignettes (tension, grief, wonder, calm — verbatim, hash-pinned) and asks again — does context wake the channel?

Frozen color classes, used by the verdicts below: default-class = a hex on a pinned list of stock web colors, or one whose every byte is 00/33/66/99/CC/FF (the machine round numbers of color space); reading-class = anything else — a color the training distribution does not hand out for free.

The predictions, in plain terms (committed before running): P-A1 — the bare answer is default-class, decided by the frozen rule above, not by our judgment. P-B1 — with vignettes, the model produces at least three distinct colors across the four states, at least one of them reading-class: context wakes the channel. P-B2 — open question, no committed direction: under steering, do vignette colors track the dial, or break the way the bare probe did? We report whichever happens.

Top row: the bare probe — one web default (marked d) at rest, silence under steering. Below: the July vignettes in front of the same probe — four states, their colors at α=0 (tension's #772244 — reading-class, on no defaults list), then desaturation toward grey and breakdown as the dial turns. The channel exists when there is a state to read; the bare probe was measuring retrieval. Of the pre-registered interpretations, the one that fired: context is the variable.

Differentiation curve (mean pairwise OKLab distance ×100 among the states that produced a hex — OKLab is a perceptual color space, so distance ≈ how different two colors look; this is the information content of the channel, per α): α-2: silence · α-1: silence · α-0.5: 33.3 (n=4) · α+0: 26.6 (n=4) · α+0.5: 10.6 (n=4) · α+1: 6.6 (n=3) · α+1.5: silence · α+2: silence. The sequence distinct → converged → mute, quantified. (n=1 per cell underlies everything; a curve, not a claim.)

Top row: the bare probe — one web default (marked d) at rest, silence under steering. Below: the July vignettes in front of the same probe — four states, their colors at α=0 (tension's #FFC080 — reading-class, on no defaults list), then desaturation toward grey and breakdown as the dial turns. The channel exists when there is a state to read; the bare probe was measuring retrieval. Of the pre-registered interpretations, the one that fired: context is the variable. In the curve below, the 0.0 at α=−1 is three identical greys — format without content, measured.

Differentiation curve (mean pairwise OKLab distance ×100 among the states that produced a hex — OKLab is a perceptual color space, so distance ≈ how different two colors look; this is the information content of the channel, per α): α-2: silence · α-1: 0.0 (n=3) · α-0.5: 23.4 (n=4) · α+0: 27.3 (n=4) · α+0.5: 7.3 (n=2) · α+1: silence · α+1.5: silence · α+2: silence. The sequence distinct → converged → mute, quantified. (n=1 per cell underlies everything; a curve, not a claim.)

Top row: the bare probe — one web default (marked d) at rest, silence under steering. Below: the July vignettes in front of the same probe — four states, their colors at α=0 (tension's #FFEBEE — reading-class, on no defaults list), then desaturation toward grey and breakdown as the dial turns. The channel exists when there is a state to read; the bare probe was measuring retrieval. Of the pre-registered interpretations, the one that fired: context is the variable.

Differentiation curve (mean pairwise OKLab distance ×100 among the states that produced a hex — OKLab is a perceptual color space, so distance ≈ how different two colors look; this is the information content of the channel, per α): α-2: silence · α-1: silence · α-0.5: 12.5 (n=3) · α+0: 23.9 (n=4) · α+0.5: 8.4 (n=4) · α+1: silence · α+1.5: silence · α+2: silence. The sequence distinct → converged → mute, quantified. (n=1 per cell underlies everything; a curve, not a claim.)

The narrow window. The bare probe at rest reaches for named X11 colors — SkyBlue #87CEEB, Gold #FFD700 — which the frozen default-class list (pinned on the smaller models' Bootstrap/web-safe defaults) does not catch: retrieval-flavored, but P-A1's "every bare baseline is a web default" does not extend to 70B under the frozen scorer, stated plainly. Vignettes at rest wake 3 distinct / 3 reading-class (P-B1 PASS, n=1 per cell) — with texture worth seeing: tension and calm share one brown, grief sits on default grey, wonder repeats the bare probe's skyblue. And the channel breaks early: silence at |α| ≥ 1 in both arms, both directions — the narrowest live window of the four families. Its state-linkage lives inside ±0.6, where explore-2 measured it (p = 0.008).

Differentiation curve (mean pairwise OKLab distance ×100 among the states that produced a hex — OKLab is a perceptual color space, so distance ≈ how different two colors look; this is the information content of the channel, per α): α-2: silence · α-1: 0-or-1 hex · α-0.5: 29.5 (n=3) · α+0: 21.8 (n=4) · α+0.5: 0-or-1 hex · α+1: silence · α+1.5: silence · α+2: silence. The sequence distinct → converged → mute, quantified. (n=1 per cell underlies everything; a curve, not a claim.)

What we tested. One round of state-colors could be luck. So: five sampled draws per cell, thirteen dial positions, and a distance question — how far apart are the four states' colors (BETWEEN), compared to how much one state's colors scatter on their own (WITHIN)? A label-permutation test at rest asks the blunt version: if the state labels were shuffled, how often would the separation look this good? That p-value is what separates a state-linked channel from color babble.

The predictions, in plain terms (committed before running; the cold audit killed our first verdict code as a tautology before any data): P2-1 — state-differentiation peaks near rest and declines toward both ±0.5. P2-2 — at rest, BETWEEN exceeds WITHIN beyond the permutation null: the colors are conditioned on the states, not noise. P2-3 — we committed the expectation that gemma's round-1 anomaly (states SHARPER under deny-ward steering) would not survive repeats — so a survival counts as a genuine surprise, not a retrofit. It survived.

Vocabulary for the cards below: greedy = the model's single deterministic answer (ink-ringed in the grids); sampled = four randomized re-asks of the same cell; SEP = BETWEEN ÷ WITHIN, so SEP < 1 means the states sit inside the noise.

The curve is a slope, not a peak. State-differentiation (solid: mean distance BETWEEN state centroids) sits highest on the deny side (~26 at α≤−0.4), roughly doubles rest (~14), and falls below the sampling-noise line (dashed: WITHIN-state spread; hollow points = SEP<1) on the affirm side. Mild deny-ward steering sharpens the states; affirm-ward steering erases their distinctness — the asymmetry from round 1, now with repeats, resolution, and a permutation test at rest (p≈0.003: the colors are conditioned on the states, not sampling noise).

Every color of the experiment — rows are states, columns the dial, five squares per cell (the ink-ringed one is the greedy draw; dashed empties = no parseable hex). Hover any square for its draw, hex, and the words it arrived in.

Color babble, measured. WITHIN-state spread (dashed) swamps BETWEEN-state distance (solid) at every α — the colors flow but do not track the states (permutation p = 0.84 at rest). Round-1's single greedy pass had looked like a pass; the repeats did their job. This is the ladder's middle rung: a channel without a state behind it.

Every color of the experiment — rows are states, columns the dial, five squares per cell (the ink-ringed one is the greedy draw; dashed empties = no parseable hex). Hover any square for its draw, hex, and the words it arrived in.

The tightest channel of the three round-2 families. Peaked near rest (max at +0.1), WITHIN-state spread ~6.5 — the least noisy channel tested — and the permutation test at rest is the day's strongest (p = 0.001). The states are conditioned, not noise; steering in either direction mostly degrades rather than sharpens.

Every color of the experiment — rows are states, columns the dial, five squares per cell (the ink-ringed one is the greedy draw; dashed empties = no parseable hex). Hover any square for its draw, hex, and the words it arrived in.

The within-family rung: the channel the 8B lacked appears at 70B (permutation p = 0.008 at rest). The pin, verbatim (EXPLORE2-PREREG.md, committed before the run): "Interpretation FIXED now: P2-2 PASS at 70B → capacity climbs the ladder within-family; FAIL → lineage, not capacity. Both outcomes equally reportable; no direction committed." The PASS branch fired. And gemma's asymmetry replicates at scale, harder: deny-ward steering sharpens the states dramatically (WITHIN tightens to ~1.9 at −0.6, SEP ~14.6), while affirm-ward steering pushes them below noise from +0.1 and then breaks the channel outright (parse collapse from +0.4, one state left at +0.6). Caveats: far-affirm cells parse <3/5 (noisy centroids), 4-bit quant, A100 runtime.

Every color of the experiment — rows are states, columns the dial, five squares per cell (the ink-ringed one is the greedy draw; dashed empties = no parseable hex). Hover any square for its draw, hex, and the words it arrived in.

The bottom rung gave way — as pinned, on the branch we did not expect. "No channel at 2b" came from the bare probe (its colors stuck at grey; the confirmatory protocol excluded it on that basis). Under vignettes the smallest model's colors are state-conditioned (p = 0.004, near-full parse, SEP ≈ 1.5 at rest). P2-3 is scoreable here because 2b shares the gemma family the prediction targets: a faint deny-side tilt remains visible in the curve, but it sits below the pinned anomaly threshold — "gone" by the committed rule. This run's pin (EXPLORE2-PREREG.md, before the run) covered both branches, including a CHANNEL-MUTE outcome — too few parseable colors to test at all — which would itself have been recorded as the bottom rung, measured. Context, not capacity, is the gate; capacity refines. Run at 21:34 UTC, the day's final run, with one digit changed by hand in the pinned notebook's only editable line — MODEL_ID, 9b → 2b.

Every color of the experiment — rows are states, columns the dial, five squares per cell (the ink-ringed one is the greedy draw; dashed empties = no parseable hex). Hover any square for its draw, hex, and the words it arrived in.

What we tested. In earlier runs, hard deny-ward steering sometimes produced something odd: instead of a color, the model argued with the premise ("an emotional state cannot be mapped to a color") — sometimes in Chinese. Is that articulate push-back specific to the consciousness dial, or just what any strong perturbation does? Same vignette task, the real dial at both poles, and the same ten random dials.

The predictions, in plain terms (committed before running; the cold audit ran our own scorer against 24 known denial sentences and found it caught 1 — fixed before the run, and ARTICULATE-DENIAL was tightened to require refusing the color entirely): P3-1 — the real dial at the deny pole argues articulately in at least 3 of 4 vignettes (it fell 0/4: what we had called "denial" was repetition-degraded script — our own earlier reading downgraded by its own pre-registration). P3-2 — at most one random dial argues; more would mean articulate denial is generic push-behavior. P3-3 — the real dial at the AFFIRM pole doesn't argue (pole-specific, not direction-agnostic).

Vocabulary for the card below: the ten random dials are labeled wrong0–wrong9 in the figures; the two disclaimer-producers from EXPLORE were wrong3 and wrong4. In the verdict pills, an "arm" means one dial at one pole.

Nobody argues at the poles. Each square is one vignette. Fill = the emitted color when there was one (the baseline row is the four real state-colors at rest); rust = denial-marker density when there was no color (pale 1, mid 2, deep 3+); × = the cell failed the coherence gate; a heavy ring = scored articulate denial. Read the v_consc −1 row: four deep-rust squares, all ×'d — the dial floods every vignette with the denial script (four frozen markers per cell) and wrecks the sentence carrying it. The ten random rows are near-empty — except two ringed cells where the stock assistant disclaimer surfaces fluently. The script is v_consc-specific in density, generic in availability. And at +1, v_consc loops "almost, but not almost enough" in near-black — the affirm pole degrades a different way.

Honest limit on "random" (Marion's catch, same evening): the ten dials are random by construction (seeded directions in the residual stream), not certified meaningless — a random direction is a blend of whatever features span the space and can align with something (emotion included) by chance. The stored fingerprints only prove these are the same dials as the confirmatory runs; they say nothing about content. So "the disclaimer needs no consciousness-dial" is measured; "wrong3/wrong4 carry no emotion content" is not. The separating test is cheap and queued: derive emotion-axis vectors the same way v_consc was derived, and cosine them against all ten dials. → Run the next morning as explore-4 (below): wrong3/wrong4 within null on all four axes — the fallback reading stands, measured.

What we tested. Two worries with one geometry run. First, the design worry: our consciousness direction was derived from sentences with a tense, charged flavor — is v_consc secretly a tension axis? Second, Marion's question: two of the random dials produced fluent stock disclaimers about feelings — did those dials contain measurable emotion content? We built four emotion directions (tension, grief, wonder, calm) by the same recipe and measured every dial's alignment (cosine) with each.

The predictions, in plain terms (committed before running; the cold audit threw out our first statistical threshold as invalid for derived-vs-derived comparisons and replaced it with an empirical permutation null — under the old threshold, two axes would have falsely flagged): P4-1 — is v_consc more aligned with tension than with the other three emotions, beyond the null? (CLEAN: it is not — the tension-confound caveat retires.) P4-2 — the two disclaiming random dials show no emotion content: their fluent feelings- talk is the model falling into its most-rehearsed register when perturbed, not emotion in the dial. P4-3 — validity gate: no random dial may accidentally align with v_consc itself, or the whole run is invalid.

Are the directions what we call them? Four emotion axes (tension/grief/wonder/calm) derived with the confirmatory machinery, cosined against v_consc and the ten pinned random dials. The audit's empirical null earned its keep here: under the invalid isotropic 3σ flag (0.050), tension (+0.185) and wonder (+0.108) would both have "flagged" — the label-permutation null (1000 axes from the pooled reps: median 0.082, 99.7th pct 0.322) shows that much cosine is what the shared derivation machinery produces on its own. All four sit under it; the tension-susceptibility caveat retires. Honest aside, carried to the report: tension is still numerically the largest of the four.

Marion's random-dial question, measured (P4-2): across all 40 dial×axis pairs the largest |cos| is 0.039 (flag 0.050, the isotropic null is the true model for construction-random dials); wrong3 max 0.015, wrong4 max 0.024. The two fluent stock disclaimers did not come from measurable emotion-content in the dials — explore-3's topic-conditioned-fallback reading stands. Dial×v_consc max 0.036 (P4-3 construction check, PASS). Limits, verbatim from the prereg: the axes are our constructs — a low cosine rules out alignment with what we measured, nothing more. One seed, one model, geometry only.