# Methodology — Sandbox Walk-Forward Backtest

**Status:** v1, 2026-04-28 · NEXUS COMMAND THv2 lane 48
**Engines:** `engine/strategy-dsl.js` (`dsl-2026-04-28-v1`) + `engine/sandbox-executor.js` (`sb-2026-04-28-v1`)
**Surface:** `pages/sandbox.html`
**Ledger contract:** every closed trade is emittable as `claim_type='sandbox-backtest'` (lane 16, schema `fl-2026-04-28-v1`).

This document is the public methodology footer for the `/sandbox` UI. It is intentionally exhaustive — per the T0 scientific-rigor charter, every backtest result must surface its assumptions, costs, and confidence intervals.

---

## 1. Strategy DSL grammar

The DSL is line-oriented. One directive per line. Comments start with `#`. Keywords are case-insensitive.

```
buy when <expr>                # required: entry rule
[sell when <expr>]             # optional: explicit exit rule
[stop at -<N>%]                # stop-loss
[target at +<N>%]              # take-profit
[horizon <N>d|<N>h]            # bar horizon (default 20d)
[universe <name>]              # universe filter
[size <equal|kelly|vol-target>]
[costs <N>bps]                 # default 5
[slippage <N>bps]              # default 2
```

Expressions are boolean over comparisons (`>`, `>=`, `<`, `<=`, `==`, `!=`) joined by `and`, `or`, `not`, with parens for grouping. Identifiers resolve through the executor's `factor(name)` lookup.

**Strict by design:**
- Missing `buy when` → parse error.
- Duplicate top-level directives (e.g. two `buy when`) → parse error.
- `stop at` requires `%` suffix; bare numbers are rejected.
- Unknown keywords or trailing tokens after a statement → parse error.
- Unbalanced parens → parse error.
- Undefined factor at evaluation time → throws (no silent zero-fill).

**No `eval()`, no `Function()`** — the parser is a hand-written Pratt-style recursive descent; the evaluator is a switch over an AST node type. CSP-safe.

## 2. Walk-forward executor

**Loop:** for each ticker `t` in the universe, walk bars `i = warmup … N-1`:

1. **Resolve open position** at bar `i`: compute `holdRet = (close_i − entry_price) / entry_price`. Exit if any of:
   - `holdRet ≤ stop_pct` (stop-loss)
   - `holdRet ≥ target_pct` (target)
   - `i − entry_idx ≥ horizon_bars` (time-stop)
   - `sell when` rule fires
   Exit fill = `bars[i+1].open` (next-bar fill — no same-bar lookahead).
2. **Look for entry** at bar `i`: if no open position and `buy when` fires, set `entry_price = bars[i+1].open`, `entry_idx = i+1`.
3. **Round-trip cost**: `(costs_bps + slippage_bps) × 2 / 10000` is subtracted from each closed trade's PnL.
4. **EOD**: any dangling open position is force-closed at the last bar's close with `exit_reason: 'end-of-data'`.

**No lookahead.** Every signal evaluation reads only bar `i`'s factor snapshot; entries/exits fill on bar `i+1`.

## 3. Risk metrics

| Metric | Formula | Notes |
|---|---|---|
| Sharpe | `mean(ret) / std(ret) × √252` | annualised; null on n<2 or std=0 |
| Sortino | `mean(ret) / downsideStd(ret) × √252` | downsideStd = RMS of negative returns; null when no negatives |
| Calmar | `(mean(ret) × 252) / |maxDD|` | annualised return ÷ max drawdown |
| Max drawdown | trough-to-peak on equity curve | reported as negative number |
| Hit rate | `wins / total` | trade-level |
| Profit factor | `Σwins / Σ|losses|` | ∞ if no losses; null if no trades |
| Expectancy | `mean(pnl_pct)` | per-trade |
| Kelly fraction | `W − (1−W)/R` where `R = avgWin/avgLoss` | clamped to [-1, 1]; null when n<5 |

## 4. Confidence intervals (T0 scientific-rigor)

Every reported Sharpe and hit-rate carries a **bootstrap 90% CI** (5th/95th percentiles), computed by resampling per-trade returns 500 times with a deterministic Mulberry32 RNG (seed=42). Sample sizes < 5 trades return `null` — there is no calibration on a sample of one.

## 5. Cost model

Two parameters, both expressed in basis points and applied per-side:

- **`costs_bps`** — broker commissions, exchange fees. Default 5 bps.
- **`slippage_bps`** — adverse fill on market orders. Default 2 bps.

Round-trip cost = `(costs + slippage) × 2 / 10000`. Subtracted from each closed trade's PnL. Caveat surfaced when both are 0 ("backtest will be optimistic").

## 6. Tax assumptions

The sandbox **does not** model tax. Tax-aware overlay is the responsibility of `engine/tax-aware-sizing.js` (lane 35). The methodology footer surfaces this explicitly: short-term realised gains are assumed; long-term holding-period adjustments and harvest opportunities are out of scope for the in-UI backtest.

## 7. Small-sample caveats

Per `feedback_silent_failures.md`, the executor surfaces a `caveats[]` array on every report:

- `n < 30` → "small-n: wide CIs, treat metrics as indicative"
- `n < 5` → "insufficient sample: kelly/CI/sortino likely null"
- Max DD < −50% → "extreme drawdown — consider tighter stops"
- Skipped tickers (eval errors / missing bars) are listed by reason

The UI paints these as warning chips beneath the stat panel.

## 8. Templates

Seven pre-built strategies are bundled in `DSL.TEMPLATES`:

| Key | Thesis |
|---|---|
| `pead` | Post-earnings announcement drift |
| `mom1m` | 1-month momentum |
| `mom3m` | 3-month momentum + score gate |
| `vol-target` | Vol-targeted entry on quality names |
| `tsmom` | Time-series momentum (Moskowitz/Ooi/Pedersen) |
| `pairs-trade` | Z-score pairs entry/exit |
| `custom-factor` | Empty scaffold for user-defined factor |

All seven parse cleanly under the strict grammar (test #28 enforces this).

## 9. Lane integration

| Lane | Module | Sandbox integration |
|---|---|---|
| 16 | `engine/forecast-ledger.js` | `SBX.toLedgerClaims(report)` → `claim_type='sandbox-backtest'` |
| 27 | `tests/_run-factor-cohort.js` | factor-cohort report metrics use the same harness primitives (sharpe, IC, hit-rate) |
| 32 | `tools/replay/replay-pipeline.js` | demo dataset = `data/replay-resolutions-2026-04-28.json` (synthesised → bar series) |
| 17 | `engine/tax-aware-sizing.js` | `size kelly|vol-target` directive — sizing applied post-hoc |
| 35 | `engine/best-actions-now.js` | sandbox-backtest claims feed best-actions ranking when ledger is shared |
| 7 | `engine/calibration-metrics.js` | report's `confidence_intervals` use the same bootstrap convention |

## 10. Replay claim contract

Each closed trade emits a ledger claim with this shape (sample):

```json
{
  "claim_type": "sandbox-backtest",
  "asof": "2026-01-15",
  "ticker": "AAPL",
  "cls": "equity_us",
  "horizon_name": "20b",
  "expires_at": "2026-02-12",
  "predicted_probUp": 0.62,
  "predicted_tier": "Buy",
  "methodology_hash": "dsl-<sha-of-strategy-text>",
  "realized_return": 0.041,
  "realized_up": 1,
  "source": "sandbox-executor-v1"
}
```

Caller is responsible for hashing the strategy text (DSL source → hash) so identical strategies produce identical claim IDs across runs.

---

**References:**
- López de Prado, *Advances in Financial Machine Learning*, ch.7 (purged-k-fold + walk-forward)
- Moskowitz, Ooi, Pedersen, "Time Series Momentum" (2012) — TSMOM template
- Sharpe, "The Sharpe Ratio" (1994); Sortino, "Downside Risk" (1991)
- Kelly, "A New Interpretation of Information Rate" (1956)