Backtest Discipline
Technical lessons drawn from the preserved case studies and research machinery behind Backtest Discipline.
The case studies show what happened in individual experiments. These notes pull out the engineering lessons that are easier to miss inside a result: executable specifications, provenance, data contracts, deterministic evaluation, and the ways software can quietly change the question being tested.
Each note starts from a concrete preserved case and then generalizes only as far as the evidence supports. The first one grows directly out of Benchmark 004, where reproducing an external notebook exposed disagreements between its written description and the code that actually produced the result.
Strategy discovery is closed. These are explanations of completed work, not new experiments or trading signals.
Benchmark 004 reproduced an external channel-breakout notebook exactly enough to expose a harder question: when the written explanation and the executed program differ, which object are you actually testing?
Worked example: Benchmark 004 — independent reproduction
Read the note