[08] What I tried
[08] · INVESTIGATION

What I tried

Every attempt, in the order I made it — what it was, what came back, and whether it was worth the time.

Scorecard

attempts50in seven phases
worth it24result or trust
dead end4tested, failed
lesson12went wrong usefully
open4not finished
not done6on purpose
worth it produced a result I would defend, or made later results trustworthy dead end tested and failed, or could never have worked lesson went wrong in a way that changed how everything after it was done open not finished — still a live lead not done left out deliberately, with the reason

In the order I did it, over a few days in late August 2026. Phases: 1 the idea · 2 transcription · 3 the lab · 4 Elgar's drawings · 5 the toolkit · 6 language models · 7 the long hunt · not done.

Worth the time

  1. Transcribing it myself and validating it — 82/87 against a published reading, p < 5×10⁻⁵. Everything else stands on this.
  2. The orientation χ² — one key-free number that killed my own first idea (p = 0.0003).
  3. Matching solver effort on both sides — the difference between a false retraction and z = −7.5.
  4. Reading the literature — Hauer (2021) and Wase (2023) had already shown the plain-substitution result.
  5. Measuring the instrument before widening the search — the bake-off and N* showed the language model mattered more than the search size.
  6. Cross-checks and planted messages — they are what make a negative result mean something.

Not worth the time

  1. Deciphering by hand under a regularly turning key — the direction counts rule it out without any key.
  2. Running the solver at unmatched effort — it produced one false alarm and one false retraction.
  3. Tiny hand-written language models — they can rank languages, never decide one.
  4. Compressed English in a large search — past about 414 candidates, coincidence wins.
  5. Worrying which side of the cup names the direction — every convention is a relabelling.
  6. Generic technical English as the target for what would be a personal letter.

Phase 1 · The idea

1.1 A snowflake key that turns — snowflake.html worth it

Did: built a one-page tool — the 24 cells as a snowflake around a hub, an A–X fill without J and Q, and a key that turns one arm forward or backward after every letter, for enciphering and deciphering.

Gave: the frame for the whole project. The glyph set really is a polar grid (Elgar drew it that way), and every later tool is this page generalised. [06]

1.2 Deciphering the note by hand under the turning key dead end

Did: in the draw.io diagram, deciphered all three lines turning forward and turning backward, key restarted at each line.

YDIA WGXF RHCL TWID GVYF ZMRP IOXB M (forward) YAKD WIWC RYDY TOKF GHVC ZKPN IXWU M (backward)

Gave: gibberish both ways — and the later engine reproduces both blocks letter for letter, so it was not a slip. It could never have worked: a regular turn forces flat direction counts, and Dorabella's are not flat (3.2).

Phase 2 · Transcription

2.1 Transcribing the note myself, glyph by glyph worth it

Did: read all 87 glyphs off the note in draw.io as arcs+direction, with the row lengths 29 / 31 / 27 fixed.

Gave: a transcription that agrees with a published 1:1 reading at 82 of 87 positions with zero letter collisions; a shuffled null averages 28.7 and never reaches 82 in 20,000 tries. [05]

2.2 Flagging nine ambiguous glyphs instead of guessing worth it

Did: wrote both options wherever I could not decide — (2ne/e) — then scored all 2⁹ = 512 combinations for best 1:1 fit.

Gave: the resolved reading (82/87, up from 79/87 for the first-pass choices) and uncertainty that is on the record rather than hidden.

2.3 The “corrected” reading lesson

Did: took five further re-reads from the published solution itself, which brings agreement to 87/87.

Learned: that is circular by construction. It is kept, labelled, and never used to test the solution it came from.

2.4 “Side open or side closed?” lesson

Did: worried in the diagram about whether a glyph's direction is named by its open side or its closed side.

Learned: it does not matter. Every naming convention is a global relabelling; all 16 offset × mirror variants give bit-identical statistics, and the solver absorbs the rest. It matters only for drawing the glyphs.

2.5 “Line by line or complete? Reversed?” worth it

Did: tested whether the line breaks are structural, and later proved the search covers both readings.

Gave: the breaks look like hand-wrapping (boundary pairs are ordinary ones, p = 0.023, weak), and rowcheck.py shows all 2,460 one-string streams sit inside the 9,835 the three-row engine enumerates. Every reading order — reversed included — is searched.

Phase 3 · The Python lab

3.1 Key-free structure tests against real English worth it

Did: measured index of coincidence, entropy, distinct symbols, doubles and repeated n-grams on Dorabella and on 400 English samples of 87 letters.

Gave: Dorabella sits inside the English range on every one (12th–74th percentile) and its frequency profile fits English at the 30th percentile. The popular claim that its statistics are anomalous does not survive calibration. [07]

3.2 The orientation χ², with a 40,000-run Monte Carlo worth it

Did: counted the eight directions (10 8 9 16 15 3 4 22) and simulated 40,000 rotating-key ciphertexts of English.

Gave: χ² = 26.6, p = 0.0003 — a regular rotating key is ruled out, with no key, solver or corpus involved. The single most robust number in the project, and it killed my starting idea.

3.3 Mirrored 180° pairs open

Did: counted consecutive same-ring glyphs facing opposite ways.

Gave: 10 against 5.2 by chance (p = 0.03). No cipher model I simulated reproduces it (3.11) — the most interesting single number in the analysis, and unexplained.

3.4 Attack, run 1 — 20 restarts, 30 controls lesson

Did: annealed a substitution on Dorabella and on 30 true-English ciphertexts, 20 restarts each.

Learned: Dorabella fell below every control — it looked decisive, but the minimum of 30 samples is not a stable statistic; the English left tail is long.

3.5 Attack, run 2 — 500 restarts against a 30-restart baseline lesson

Did: gave the seven best transforms 500 restarts, but the 100-sample English baseline only 30.

Learned: unsound in two directions at once — the baseline was under-solved (its floor dragged down to −4.900) while Dorabella was over-solved. It landed “inside the range, about the 6th percentile” and I retracted run 1. The retraction was wrong.

3.6 Attack, run 3 — matched effort worth it

Did: 100 restarts × 12,000 iterations on both sides, 40 English controls.

Gave: every reading order at the 0th percentile, z between −7.5 and −9.6, while the solver recovers a median 87/87 letters on the controls. A plain substitution of English is excluded — the number to quote, and the same conclusion as Wase (2023).

3.7 Every reading order × every regular rotation dead end

Did: solved 6 reading orders, 7 rotation schedules × 7 step sizes, and reversed-then-derotated variants (hypotheses.py).

Gave: nothing that reads. Expected after 3.2, and now measured.

3.8 Cribs — DORABELLA and friends worth it

Did: a crib tester that checks whether a word's letter-repetition pattern can sit at a position before any solving, then solves with it locked.

Gave: a cheap, hard filter. DORABELLA (repetition pattern 012345663 — the double L, the returning A) fits nowhere in the as-written reading, nor in any of the other six reading orders under a static key — ruled out before a solver runs. No crib-based reading is recorded.

3.9 Six small language models lesson

Did: bigram models for English, German, French, Italian, Latin and Spanish from about 1.7 KB of hand-written prose each (34–48% bigram coverage).

Learned: they rank candidates, they cannot decide one. They happened to put Latin first and English second — Hauer et al., with 100M characters per language, put Latin first too — a coincidence worth checking, not a result. It led to the proper German model (6.2).

3.10 Periodic IC and Kasiski worth it

Did: coset IC for periods 1–22 against 400 shuffles each, and Kasiski spacings.

Gave: no period is elevated — no repeating key of length 2–22.

3.11 Nine cipher models, simulated and compared worth it

Did: simulated nine mechanisms an 1897 amateur could use, 320 runs each, and asked where Dorabella falls in each one's own output on five diagnostics.

Gave: plain English substitution 4.7%, regular rotating key, Wheatstone cryptograph and hoax 0.0%. The top four tie on evidence and are ordered only by my priors — and the mirrored-pair count sits outside all nine. [10]

Phase 4 · Elgar's own drawings

4.1 The notebook messages worth it

Did: worked through the 1920s notebook page and wrote the plaintext in: MARCO ELGAR, A VERY OLD CYPHER, DO YOU GO TO LONDON (TOMORROW?).

Gave: the only known plaintext in his alphabet. It reads straight off with the key on the page — the alphabet is not the secret. [03]

4.2 The clock-face circles, and my overlay worth it

Did: studied the notebook's circle diagrams (M1–M4) and traced the glyph shapes into one of them.

Gave: the polar-key reading — radius = arcs, angle = direction — grounded in Elgar's own drawings rather than imposed on them.

4.3 Compass roses dead end

Did: looked at 8-, 16- and 32-point compass cards as a model for the eight directions.

Gave: a good picture for the idea, but an analogy, not evidence: nothing testable came out of it that the clock-face diagrams do not already give.

4.4 The Crystal Palace fragment (1886) open

Did: collected the programme page and cropped the marginal glyphs.

Gave: not attacked yet. Eighteen characters in the same alphabet, eleven years earlier — if one key fails on both texts, that is evidence about the key.

Phase 5 · The browser toolkit

5.1 Nine offline pages on one shared engine worth it

Did: rewrote the snowflake as a general polar-key engine (core.js) shared by every page, with core.test.js: 4,049 assertions, 3,360 encipher/decipher round-trips across the whole symmetry group. No server, no build step needed to use it, works from file://.

Gave: tools that cannot drift from each other. On its own the toolkit found nothing new — its value is that everything the lab claims can be replayed in a browser.

5.2 Decipher in phases, with a two-stage key hunt worth it

Did: transcribe → sequence → unwind the key motion → substitute → score against a calibrated baseline → hunt; the hunt hashes identical streams (a measured 50–56% saving) and re-solves only the leaders at full effort.

Gave: every step inspectable, and the rotation space swept at browser scale. No reading came out of it.

5.3 Reproducibility recipes worth it

Did: a recipe format with a deterministic Level 1 — ciphertext + reading order + key motion + 24-cell key gives the plaintext by arithmetic — and a stochastic Level 2 that re-runs the annealer from its seed.

Gave: a claim that fails Level 1 is not a solution, whatever it scores; one altered key cell fails loudly with the divergence index. Anyone's claim can be pasted in and checked.

5.4 The transposition bench — stated rules only worth it

Did: columnar, double columnar, rail fence, five routes and the turning grille as exact permutations you apply, plus a one-pass sweep of the fixed catalogue as a screen.

Gave: a guard as much as a tool: it refuses to search for a rearrangement, because at 87 characters that manufactures English by chance.

5.5 String experiments — undo the mirrored pairs, split at the dot open

Did: a page to chain rearrangements (reverse, swap adjacent pairs, split at the dot, decimate, halves…) and unwind a moving key after them, with a leaderboard that states how many chains were tried.

Gave: the presets with real motivation — a mirrored pair as a unit written in the wrong order, the dot as the only punctuation — are still the most principled untested ideas. No result from them is recorded.

5.6 Continuous auto-solve with a bandit rule lesson

Did: a sweep meant to be left running, choosing which candidate to re-solve with a bandit rule.

Learned: it fixated — 267 of 319 rounds on one candidate. “Best score so far” only rises with more rounds, so the bandit rewarded whatever had been pulled most. Replaced by successive halving with equal budgets.

5.7 Templates and saved investigations worth it

Did: a template to assert plaintext at positions, and a session file holding transcription, transform, key, template and the solver's work.

Gave: a guess becomes a test: pinning a right guess costs little score, a wrong one costs a lot.

Phase 6 · Better language models

6.1 English from technical text lesson

Did: built the first English n-gram model from Python docstrings and package docs blended with hand-written prose.

Learned: its letter frequencies are about 11 points off standard English, mostly H and W; a Victorian personal note could be under-scored. It stayed for the matched-effort result (which it passes: controls solve to 87/87) but was replaced as the default target.

6.2 A real German model, with held-out controls worth it

Did: a 3+4-gram model over 2.5 M letters of 19th-century prose (Werther, Effi Briest, Buddenbrooks, Die Verwandlung, Zarathustra, Faust I), with controls from a held-out 15%.

Gave: German on an equal footing with English. Held-out mattered: in-corpus controls read 0.43 SD too high, which would have made every candidate look worse. On German text it separates at 11.3 noise-SD.

6.3 Period correspondence, and the model bake-off worth it

Did: a 3 M-letter corpus of period letters (Carroll, Dickens, George Eliot, Darwin, Austen, Keats, plus model letters, a letter-writing manual and merchant correspondence), filtered to sentences that read like letters, and a bake-off: the same plaintext judged by each model.

Gave: at this search size the bar is 5.8 noise-SD. Generic English misses it for a personal letter (4.7) and business letters (4.1); the correspondence model clears both (8.0, 7.7). The earlier searches could not have detected the message in either register.

6.4 Compressed English as a shorthand proxy dead end

Did: a vowel-reduced English model to test the private-shorthand hypothesis.

Gave: a correct answer beats noise by only 0.135 (separation +0.12), so past about 414 candidates coincidence wins. Useless in a large search; dropped from the defaults, kept for small targeted runs.

6.5 The search budget N* worth it

Did: measured, per language, how many candidates a search can test before its best coincidence outscores a genuine solution.

Gave: N* ≈ 1.2×10¹⁶ for English and German. At the full 326 M candidates the truth still wins by 0.57 — so the long hunt is worth running, and a positive English or German result would mean something. [09]

Phase 7 · The long hunt

7.1 dorabella_hunt.py — a months-long search open

Did: a single-file, stdlib-only server search over 15 cipher families and about 108 million candidates, with checkpoints, a live dashboard and a verdict built on a permutation null, best-of-N and Šidák.

Gave: an honest instrument that is slow on purpose: the whole space is about 2.5 years on four cores, and it samples every family in turn rather than finishing one. No solution is recorded; the expectation, written into its manual, is “almost certainly nothing”. [09]

7.2 Proportional scheduling lesson

Learned: scheduling families by fraction explored sounds fair and is not — the snowflake family took 798 of the first 800 slots and decimation never ran. Replaced by round-robin.

7.3 Short streams judged against long controls lesson

Learned: the null-removal family halves the text; judged against 87-character controls it swept the leaderboard at z = +3.07 on the first live run, purely because 43 characters score higher than 87. Calibration is now built per length.

7.4 Deepened leaders judged against a cheap baseline lesson

Learned: the hunt once reported “solution” for English and German at the same time — definitionally impossible. Leaders re-solved at twelve times the effort were still judged against a one-times baseline. Calibration now matches language, length, restarts and iterations.

7.5 How much effort per candidate lesson

Learned: the thin survey (8 × 4,000) recovers a median 8% of letters on planted English; the deepened pass (96 × 8,000) recovers 95%. Effort is two numbers, not one: cutting iterations from 6,000 to 4,000 cost 19 points of recovery even at 96 restarts.

7.6 A process that looked dead, a page that showed nothing lesson

Learned: the status file was written from the result loop, so a long deepening pass looked exactly like a dead process — it now ticks on its own thread. And mkstemp leaves files 0600, so the dashboard (running as www-data) showed nothing while the hunt worked perfectly — it now chmods 0644.

7.7 Cross-checking the server engine against the browser's worth it

Did: ran the Python hand-port and core.js over the same transforms and compared the unwound streams cell by cell.

Gave: 4,107 transforms, identical — so anything the server finds can be reproduced in the browser decipher tool.

7.8 Planted messages for the new disc families worth it

Gave: ringspin, keyspin and autokey each recover a planted message at 100% — proof the families are genuinely searchable, not just enumerated. [06]

7.9 Word watch, with the chance figure beside it worth it

Gave: watching for ELGAR, DORA, MUSIC… turns up hits whether or not anything is there. A test run over 96 candidates found AND twice where chance alone predicts 4.23 — a ratio of 0.47×. Printing the chance figure is what keeps the eye honest.

7.10 Crash safety — the kill -9 test worth it

Gave: after a hard kill mid-run, the restart carried all prior work, reused 34 baselines, and kept 25 leaders and the whole ledger. A search meant to run for months has to survive a power cut.

Deliberately not done

← 07Measurements09 →The long hunt