# Methodology — Factor Orthogonalization (THv2)

**Status:** Shadow-path layer. Canonical orchestrator unchanged.
**Version:** 1.0 — 2026-04-28
**Lane:** NEXUS COMMAND THv2 lane 5 (factor orthogonalization + scientific-rigor scaffolding)

---

## Why orthogonalize?

THv2 publishes a 30+ factor score. If those factors share substantial variance, the score is a 30-factor *label* on a much smaller-dimensional reality — confidence theater. Two consequences:

1. **Inflated "ensemble breadth"** — five momentum factors voting bullish look like five independent confirmations but are one signal counted five times.
2. **Ill-conditioned weight estimation** — covariance-based weighting (ICIR ensembles, regression composites) explodes when columns are near-collinear.

Orthogonalization decomposes the factor matrix into **statistically independent components**. The score is then computed from those components, not from the redundant raw factors.

## Three-mode toolkit

`engine/factor-ortho.js` exposes three orthogonalization modes; each suits a different use case:

### 1. PCA (`pca`)
- **What:** SVD-equivalent decomposition (Jacobi eigen-decomposition on the covariance of the standardized factor matrix). Returns top-k principal components, loadings, and explained variance per component.
- **Use:** When you want a fully data-driven projection that *maximizes captured variance per dimension*. The orchestrator-ortho shadow path uses this.
- **Numerical safeguards:** standardize before decomposition; eigenvectors are orthonormal by construction; variance components are guaranteed non-negative; tolerance check 1e-14 on off-diagonal mass.

### 2. Pivoted Gram-Schmidt (`gramSchmidt`)
- **What:** Modified Gram-Schmidt with caller-specified column priority. Earlier-listed factors keep their original direction; later ones are residualized against everything earlier.
- **Use:** When you want to preserve *interpretation* — e.g., keep "F04_mom1m" as-is so dashboards still say "momentum", then strip momentum from F05/F06/F07.
- **Numerical safeguards:** column norm tracked in `R[k][k]`; small diagonal indicates near-collinearity (downstream consumers may flag/drop).

### 3. Regression residuals (`regressionResiduals`)
- **What:** Solve `min ||y − Xβ||²` via normal equations with Tikhonov regularization (auto-applied when condition number > 1e8).
- **Use:** Marginal contribution. "After controlling for F01..F09, what does F10 actually add?" Equivalent to Fama-MacBeth regressions for surfacing whether a new candidate factor carries independent alpha.
- **Numerical safeguards:** partial-pivoting Gauss-Jordan; ridge λ = 1e-6 · trace(X'X)/p when ill-conditioned; residuals sum to zero when intercept column is included.

## Score-via-orthogonal-components

`engine/orchestrator-ortho.js` runs the **shadow** score path:

```
factorMatrix → standardize → PCA (top-k) → tanh-bounded component scores → variance-weighted sum → 0..100
```

Activation: `localStorage.THV2_ORTHO_SCORE === '1'`. While off, the canonical orchestrator runs unchanged. While on, both scores are computed and surfaced for side-by-side comparison.

Output payload:
```js
{
  score,                           // 0..100, schema-compatible
  tier,                            // S/A/B/C/D/E/F per ScoreSchema
  alphaShareByComponent: [...],    // % of |total contribution| per component
  componentScores: [...],          // raw projections
  orthogonalComponentsUsed: k,
  explainedVarianceCovered,        // cumulative EV for the k retained components
  missingFactors                   // count of NaN inputs
}
```

## IC gating (factor-ic.js)

The Information Coefficient (IC) is the Spearman rank correlation between **today's factor score** and **forward N-day return** across the universe. ICIR = mean(IC) / stdev(IC) over a rolling window.

**Archive-candidate rule:** rolling 90-day mean |IC| < 0.05 **AND** 95% bootstrap CI upper bound < 0.05 **AND** sustained for ≥ 30 consecutive days.

Per `feedback_nexus_kill_failing_experiments.md`:
- Archive candidates are **surfaced only**. Never auto-archived.
- Double-confirm + 90-day cooldown required before any factor is removed.
- Bootstrap CIs (500 iters, seeded for reproducibility) ship with every candidate.

## Scientific-rigor checks

- **Walk-forward backtest** (`tests/ortho-vs-canonical-walkforward.test.js` equivalent at `src/lib/ortho-vs-canonical-walkforward.test.js`):
  - 252-day train / 63-day test rolling windows over a synthetic universe with known factor structure.
  - Compares Sharpe, top-quintile hit rate, mean forward return, Brier on tier-derived hit probabilities.
- **Null-hypothesis test:** When forward returns are pure noise, ortho-score top-quintile hit rate must lie inside [0.42, 0.58] (loosely 0.5 ± 8pp for finite-sample synthetic universes).
- **Orthogonality check:** Every `gramSchmidt` test verifies Q^T Q ≈ I within 1e-6.

## Limitations & honest disclosures

- **Standardization changes the basis.** PCA components depend on whether factors are scaled. We standardize by default; consequence is that high-vol factors no longer dominate by virtue of unit choice.
- **Stationarity assumption.** The training-window correlation matrix is held constant over the test window. If factor-correlation regimes shift mid-window, the basis becomes stale. Re-fit cadence: 63 days.
- **Synthetic-universe walk-forward is not a real-data backtest.** Results in the findings doc are *structural sanity checks* (ortho beats random; alpha exists; Brier < 0.30). A real-universe walk-forward is queued for the next pass once snapshot ring history reaches ≥ 252 trading days of cross-sectional data.
- **No factor is auto-archived.** All gating is advisory. The `kill-failing-experiments` charter (double-confirm + 90-day cooldown) gates every removal.

## See also

- `engine/factor-correlation.js` — diagnostic scaffolding (Pearson + Spearman matrix, eigenvalues, condition number, participation ratio)
- `engine/factor-ortho.js` — three-mode orthogonalization toolkit
- `engine/orchestrator-ortho.js` — shadow score path
- `engine/factor-ic.js` — rolling IC, ICIR, archive-candidate gate with bootstrap CIs
- `engine/orthogonalize.js` — pre-existing cross-sectional residualization (compatible; not replaced)
- `docs/factor-orthogonalization-findings-2026-04-28.md` — first offline run