🧠 Research · Guide

Backtesting & Hyper-Optimization — tuning without lying to yourself

Any backtester can produce a beautiful curve; the hard part is producing one that survives contact with next week. This guide covers the platform's replay engine, the walk-forward optimizer with its out-of-sample admission gate, per-symbol tuning maps, and the honesty rules that keep every simulated number pessimistic on purpose.

The replay engine

Strategies are replayed against real exchange kline history with exact fee modeling (maker vs taker per fill), slippage assumptions on taker exits, funding where relevant, and the same triple-barrier exit semantics the live desks use — barriers are checked against highs and lows, not closes, so a wick that would have stopped you live stops you in the test. Strategy tests sweep the whole symbol universe (top-N by volume) so a strategy is judged on breadth, not on the one pair it happens to fit.

Why most backtests lie

• In-sample overfitting — tune 10 knobs on one window and you have memorized noise. The curve is a photograph of the past, not a forecast.
• Look-ahead leakage — indicators computed on unclosed candles, signals that peek one bar forward. Our indicator engine evaluates closed candles only, and ML features are built strictly causal — every algo passes a truncation-invariance check (a signal computed at bar N must be identical whether or not bars after N exist).
• Optimistic fills — assuming your limit order filled because price touched it once. The simulator requires penetration, the same rule the desks' honest paper mode uses.

Walk-forward with an admission gate

The 🧠 Hyper-Optimizer splits history into a search window and a held-out validation window. It searches knob combinations on the older window only, then runs the single best candidate — once, blind — on the recent window. The candidate is admitted only if it beats your current champion out-of-sample. No admission, no change: the optimizer is structurally incapable of installing a memorized past. Scoring balances return with drawdown and trade count, so a two-trade fluke can't outrank a robust performer.

🎯 Per-symbol tuning maps

Admitted winners are saved per strategy and per symbol — because DOGE and BTC do not share optimal spacing, and pretending otherwise is a quiet form of overfitting too. Deployed strategies and the MM desk pick up their 🎯 tuned values automatically; your ⚙ manual per-symbol overrides always outrank the optimizer (merge order: desk defaults → 🎯 tuning → ⚙ manual).

Optimizing the desks, not just algos

Uniquely, the market-making desk itself is optimizable: the session simulator replays the desk's real mechanics — lot ladders, inventory limits, cool-offs, fill penetration, session loss stops — so a search over spread/skew/vol-multiplier knobs is judged by desk-parity simulation, not by a toy approximation. What the optimizer admits is what the desk will actually do.

ML meta-labeling & causal features

The algo library includes gradient-boosted models (a pure, dependency-free implementation) used the disciplined way: as meta-labelers that size or veto a base signal rather than hallucinate direction from nothing. Features are built strictly causal and every model passes the truncation-invariance test before it may trade. When inputs are missing the model abstains — the platform-wide rule: skip, never guess.

FAQ

Related guides