# Methodology — `/track-record` (the public-miss-rate page) **Date:** 2026-04-28 **Lane:** THv2 lane 28 **Status:** Spec + implementation **Authors:** NEXUS COMMAND THv2 lane 28 **Cross-cutting charter:** `feedback_scientific_rigor_portfolio_wide.md` (T0) **Coordinates with:** lane 7 calibration metrics, lane 9 competitive matrix, lane 16 forecast-ledger, lane 24 bridge publisher, lane 26 verdict-modal attestation --- ## 1. What this page is The `/track-record` page is the **public-facing surface of the lane-16 forecast-ledger** — it publishes every score Trading Hub v2 ever rendered, with its predicted probability, predicted magnitude, predicted horizon, and what actually happened. This is the category-killer trust moat per `reports/category-killer-enhancements-2026-04-28.md` §S2: *"The only verdict modal in fintech that publishes its own miss rate."* Bloomberg / FactSet / FactorX won't publish their miss rates because brand-incompatible. THv2 wins on this dimension by being honest. --- ## 2. Data flow ``` [user scores] → orchestrator → forecast-ledger.recordClaim() │ ▼ data/forecast-ledger.json │ horizon-end → cron/resolve-forecasts → ledgerApi.resolveClaim() │ ▼ /api/track-record/get ← engine/track-record-data.js (NEW, lane 28) │ ┌───────────────────────────┼───────────────────────────┐ ▼ ▼ ▼ /track-record.html /verdict-modal?claim=… /api/audit-verify (full scoreboard) (per-claim drill-in) (attestation hash) ``` Every step is **pure-function** except for fs reads of `data/forecast-ledger.json` and the API rate-limit gate. --- ## 3. Metrics computed All metrics defer to `engine/calibration-metrics.js` (lane 7). The track-record module never re-implements; it composes. | Metric | Computation | Range | Halt threshold | |---|---|---|---| | Brier score | `mean((predicted_probUp - realized_up)²)` | [0, 1] | rolling-30d > 0.30 → calibration halt | | Log loss | `-mean(y·log(p) + (1−y)·log(1−p))` | [0, ∞) | informational | | ECE (10-bin) | `Σ (n_bin / N) · \|predicted_bin − actual_bin\|` | [0, 1] | rolling-30d > 0.10 → calibration halt | | MCE | worst-bin `\|predicted − actual\|` | [0, 1] | informational | | Sharpness | `var(probs)` | [0, 0.25] | rewards confidence | | Hit rate | `(predicted_probUp ≥ 0.5) === realized_up` | [0, 1] | StrongBuy 30d <0.50 with n≥30 → tier-cap dampening | | Wilson 95% CI | `[ŷ ± z·√(ŷ(1−ŷ)/n + z²/4n²) / (1 + z²/n)]` | — | reported alongside hit rate | --- ## 4. Scoreboards The page renders three orthogonal stratifications: ### 4.1 Per-tier scoreboard Stratified by `predicted_tier` ∈ {StrongBuy, Buy, Hold, Sell, StrongSell}. Honest-calibration shows that StrongBuy claims at 80% probability should land 80 ± CI% of the time. ### 4.2 Per-class scoreboard Stratified by `cls` ∈ {equity_us, equity_intl, crypto, commodity, fx, rates}. Different base rates → only honest comparison is per-class. Crypto's high vol means equal hit rate ≠ equal Brier across classes. ### 4.3 Per-horizon scoreboard Stratified by `horizon_name` ∈ {1d, 5d, 20d, 60d, 252d}. Calibration generally degrades at longer horizons; we publish that honestly rather than headline only the best window. --- ## 5. Reliability diagram 10-bin SVG plot (CSP-clean: no inline scripts; bins programmatically rendered): - **X-axis:** predicted probability (binned 0.0..1.0 in 0.1 steps) - **Y-axis:** realized hit rate per bin - **Diagonal:** perfect calibration reference line - **Bin markers:** `r ∝ log(n)` so visual weight reflects sample size - **CI bars:** Wilson 95% vertical bars per bin - **Click semantics:** any bin → drill-in showing claims in that probability range Forwards to `CalibrationMetrics.calibrationCurve(probs, outcomes, 10)`. --- ## 6. Hall of Fame & Wall of Shame Defined explicitly to avoid cherry-picking: - **Conviction** = `|predicted_probUp − 0.5| · 2` (range [0, 1]) - **Magnitude** = `|realized_return|` (price move size, %) - **Correct** = `(predicted_probUp ≥ 0.5 ? 1 : 0) === realized_up` - **HoF score** = `conviction × magnitude`, filtered to **correct** calls only, top-10 - **WoS score** = `conviction × magnitude`, filtered to **wrong** calls only, top-10 The Wall of Shame is the integrity test: if a fintech can't show its biggest blowups, the calibration claims are theatre. Every WoS row links to its original Verdict Modal via the lane-26 attestation hash for full reproduction. --- ## 7. Drift watch Rolling 30d / 90d windows compared against the prior 30d / 90d windows. Reports: - `window_30d` — most recent 30 days of resolved claims - `window_90d` — most recent 90 days - `prior_30d` — claims resolved 30-60 days ago - `prior_90d` — claims resolved 90-180 days ago - `brier_delta_vs_prior` — positive = worsening - `ece_delta_vs_prior` — positive = worsening - `strong_buy_30d` — n + hit_rate + Wilson CI for the StrongBuy 30d cohort ### Halt flags The drift-watch object surfaces three halt flags that the page renders as a top-of-page CALIBRATION HALT banner: 1. `brier_30d_halt` — true if 30d Brier > 0.30 (lane 7 charter rule) 2. `ece_30d_halt` — true if 30d ECE > 0.10 (lane 7 charter rule) 3. `strong_buy_floor_breach` — true if ≥ 30 StrongBuy calls in 30d with hit rate < 50% When any flag fires, the page activates the halt banner AND fires the lane-15 alerts pipeline (handled by the cron, not by the static page). --- ## 8. Cohort vs benchmark `cohortVsBenchmark(claims, benchmark)` produces a cumulative-return curve for the cohort vs SPY (or another supplied benchmark series). **Always annotated with caveats** per `feedback_scientific_rigor_portfolio_wide.md`: - gross of transaction costs - gross of slippage - gross of taxes - no position-sizing normalization - survivorship excluded - past performance does not guarantee future results This is **not a backtest of a tradeable strategy**. It is an aggregate of "if you had traded each call equal-weight". Treat as illustrative; do not extrapolate to live performance. --- ## 9. Verifiability flow Each row in HoF, WoS, drill-in, and audit trail carries `attestation_hash` from lane 26. To independently verify a row: 1. Copy the `attestation_hash`. 2. POST to `/api/audit-verify` with the hash. 3. The server confirms the claim was made at the timestamp shown with those exact inputs and that exact prediction. The attestation proves **input inviolability**, not methodology optimality. A correct attestation does not mean the methodology was correct — it means the inputs and prediction at that timestamp were not tampered with after the fact. The audit-trail JSON download (`/api/track-record/get?audit=1`) returns every claim with full forecast inputs + resolution. Anyone can re-run the whole calibration computation from this JSON locally. --- ## 10. Drill-in semantics | Click target | Filter applied | |---|---| | Per-tier row | `tier = ` | | Per-class row | `cls = ` | | Per-horizon row | `horizon = ` | | Reliability bin | `prob_bin = ` | All drill-ins call `/api/track-record/get?=` and render the matching claims with attestation links. --- ## 11. API contract `GET /api/track-record/get` | Query param | Type | Effect | |---|---|---| | (none) | — | full scoreboard payload (default) | | `cls` | string | filter to single asset class | | `tier` | string | filter to single predicted tier | | `horizon` | string | filter to single horizon | | `prob_bin` | int 0..9 | reliability-bin drill-in | | `audit=1` | flag | flat audit-trail rows | | `limit` | int | cap audit-trail rows (default 5000, max 50000) | | `since` | ISO-8601 | resolved_at lower bound | | `until` | ISO-8601 | resolved_at upper bound | **Auth:** public. **Rate limit:** 60 req/min/IP via `applyGate`. **Cache:** `public, max-age=60, stale-while-revalidate=300`. **Empty-state honesty (per `feedback_silent_failures.md`):** when no resolved claims exist, payload includes `empty_state_reason: 'no_resolved_claims_yet'` and the page surfaces this to the user — never blank-and-silent. --- ## 12. What this page deliberately does NOT do - **Does not sell.** No "see how we beat SPY by X%" copy. - **Does not claim a backtest return.** Cohort cumulative is illustrative; transaction-cost / slippage / tax caveats are *required* annotations. - **Does not hide losses.** WoS is co-equal in visual weight with HoF. - **Does not gate behind paywall.** Free + verifiable is the moat. - **Does not optimize the leaderboard.** Tier-stratification + per-class + per-horizon prevents single-number theatre. --- ## 13. Cross-lane dependencies | Lane | Status | Coupling | |---|---|---| | 7 — calibration metrics | DONE | hard dep — track-record forwards all calibration math to lane-7 module | | 9 — competitive matrix | DONE | this page realizes their category-killer positioning | | 15 — alerts pipeline | DONE | drift-watch halts fire alerts via the existing pipe (handled by cron, not by static page) | | 16 — forecast ledger | DONE | hard dep — this page consumes ledger resolved-claims | | 24 — bridge publisher | IN FLIGHT | track-record state should be published in the bridge bundle once that lane lands | | 26 — verdict modal | IN FLIGHT | this page is the directory of all verdicts ever rendered; clicking a row → opens that historical Verdict Modal | --- ## 14. Test coverage `src/lib/track-record.test.js` — 64 vitest cases covering: - module surface + halt-threshold constants - empty / malformed-input no-silent-failure semantics - per-tier / per-class / per-horizon scoreboard totals - reliability diagram delegation to lane 7 - HoF / WoS ranking (correctness, ordering, cap) - drift watch windowing + halt-flag detection (3 halt scenarios) - cohort drill-in filter combinatorics (5 filter shapes) - cohort vs benchmark with required caveats - audit-trail flat-row preservation - master `buildScoreboard` payload shape - helpers (Wilson CI, hit rate, conviction, correctness) All 64 tests pass. --- ## 15. References - `engine/calibration-metrics.js` — lane 7 - `engine/forecast-ledger.js` — lane 16 - `docs/forecast-ledger-schema-2026-04-28.md` — lane 16 schema - `reports/category-killer-enhancements-2026-04-28.md` — lane 9 positioning - `feedback_scientific_rigor_portfolio_wide.md` — T0 charter - `feedback_silent_failures.md` — empty-state honesty - `feedback_progressive_disclosure.md` — drill-in pattern - `feedback_actionable_first.md` — top-10 best/worst surfacing