Independent strategy testing
Backtest Discipline is an independent, noncommercial research publication. I test selected published strategies and explain the results, implementation choices, and limits of the evidence. All research is freely accessible.
Independent research publication · New evaluations published periodically.
Latest evaluation ·
Reproducing a published Bitcoin example showed how a one-bar timing error changed simulated wealth from 14.41× to 6,094.94×. The numerical comparison passed; the inflated ledger cannot be traded as written.
17 Aug 2017 – 4 Oct 2026
BTCUSDT · Binance Spot · daily bars
Research record
Each study asks a specific question. I explain what a reproduction matched, which acceptance conditions failed, or why a data problem stopped evaluation.
Original hypothesis validation · historical holdout
The estimate pointed slightly toward reversal, but the confidence bound stayed above zero. The frozen 2026 BTCUSDT holdout did not establish the claimed effect.
· 1 Jan – 31 Aug 2026
Read the case studyPublished-finding historical out-of-sample validation
After extreme one-minute ETH gains, the average 30-minute response was +2.99 bps, but the lower confidence bound crossed zero. The frozen criterion was not met.
· 1 Jan – 31 Aug 2026
Read the case studyHow the tests work
For developers, independent researchers, and readers who want to look beyond a performance headline.
01 · Select and specify
Identify the original source, explain ambiguous rules, and distinguish reproducing a result from testing a mechanism on another period.
02 · Fix the test
Specify the data, execution timing, costs, benchmark, and acceptance criteria before capture and evaluation. Historical testing is not live trading.
03 · Evaluate and explain
Apply the fixed criteria, explain the trade-offs and limitations, and preserve supporting artifacts. Neither a favorable nor an unfavorable verdict is the objective.
Engineering notes
Shorter explanations of the metrics, assumptions, and software decisions behind the tests.
· Engineering note
This DAR(4) validation got messy before the confirmatory run ever started: failed numerical reconstruction, a bad frozen acquisition boundary, a preserved predecessor, a successor lineage, and finally 1,686 sealed forecasts. Most of the work was research math; the platform’s job was to keep the history honest.
Read the note· Engineering note
A five-ULP match was the visible result. Underneath it were harder engineering questions: what identifies the data, where approximate equality is allowed, how a completed run stays immutable, and how to add a second research shape without creating a second maintenance path.
Read the noteBehind the publication
I built cryptobot-research to look for strategies that could survive controlled testing. After ending that open-ended search, I shifted to independently evaluating selected published strategies. Here I publish the results, implementation choices and evidence limits.
New evaluations continue. Completed experiments keep their original rules and recorded outcomes.
About the publicationNew studies and engineering notes are available through the site's RSS feed. You can also suggest a published strategy for consideration.