# Backtest review worksheet

From Backtest Discipline: https://backtestdiscipline.com/guides/backtest-review-checklist/

The guide at that address explains each group and links to a published case where the question mattered.

Use this worksheet to record what is documented about a backtest and what is missing. It is not a score. Completing it does not make a backtest valid, and an unanswered question does not make one wrong.

For each question, fill in:

- **Status:** `documented`, `unclear`, `unavailable`, or `not applicable` (give a reason)
- **Answer:** what the source says or does
- **Evidence / location:** file, page, line, table, or commit where you found it
- **Remaining concern:** what is still open

---

Backtest under review:

Source and version:

Reviewed by / date:

---

## 1. What exactly is being tested?

**1.1 Is the source pinned to a version (a commit, a paper DOI, a dated video)?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**1.2 Where the prose, comments and code disagree, which one generated the reported numbers?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**1.3 Are any rules discretionary? How were they turned into code, and what was left out?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**1.4 Is every parameter value stated, or do some have to be inferred?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 2. Which data, and is it complete?

**2.1 Which venue, instrument and bar interval? Is it standing in for something broader?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**2.2 Can the exact input be identified (a file hash, an archive version) or captured again?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**2.3 How many rows were expected and how many arrived? Were gaps filled, dropped, or left missing under a stated rule?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**2.4 Was any repair or filling rule chosen after the result was visible?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 3. Could the backtest know something it shouldn't?

**3.1 When does a signal form, and when is the trade assumed to execute?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**3.2 Was the set of instruments chosen using information from after the test started (current listings, survival, full-period data coverage)?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**3.3 When one bar reaches both the stop and the target, which is assumed to happen first?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**3.4 Which time zone defines a "day" or a trading session?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 4. What do the costs and execution assume?

**4.1 What does the cost formula actually charge (per trade, per side, per unit of turnover)? Does it include spread and slippage?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**4.2 How large is the gap between the gross and net results?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**4.3 Was the cost assumption fixed before the result, and does the conclusion survive a reasonable range of costs?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**4.4 If the strategy goes short or uses leverage, are borrowing, funding, margin and liquidation modelled?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 5. Which return, and compared with what?

**5.1 Is the headline a compounded total, an arithmetic annualized mean, a geometric annualized rate, or a log return?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**5.2 Is there a benchmark, and is it charged the same costs?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**5.3 What does the strategy hold when it is out of the market?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**5.4 Is the claim absolute ("it made money") or relative ("it beat holding the asset")?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 6. How were the parameters chosen, and when was the test fixed?

**6.1 How many variants were tried? Is the reported one the best of many?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**6.2 Was a primary variant named before the result, and can another variant replace it afterward?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**6.3 What was fixed before the test data were seen? Is "before" measured against the data download or against the market observations themselves?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**6.4 Is the test period a live forward test, a later historical period, or the same period used to develop the rules?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 7. Is the result distinguishable from noise?

**7.1 How many observations support the estimate, and do they overlap?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**7.2 Is there a confidence interval or bound, and does it allow for dependence between observations?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**7.3 Does the result hold across sub-periods, or does one stretch carry it?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**7.4 Was the threshold for "established" set before the result was known?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

## 8. What can you check yourself?

**8.1 Can you get the code, the exact data and the expected outputs? Which are missing?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**8.2 If someone reproduced the result, did they reproduce a number or validate a strategy?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**8.3 If "matching" allowed a tolerance, was it set before the comparison?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

**8.4 Are hashes being used to show more than content identity, or self-recorded timestamps to show more than the order of steps?**

- Status:
- Answer:
- Evidence / location:
- Remaining concern:

---

## Summary

Open questions to send to the author, or to fix first in your own backtest:

1.
2.
3.