Is That Strategy Backtested or Just Curve-Fit? How to Spot Over-Optimized Strategies

What Curve-Fitting Actually Is

Curve-fitting means a strategy's parameters were adjusted — deliberately or not — until the equity curve looked smooth on the same data used to judge it. The model has memorized noise, not learned signal. Every historical dataset contains random sequences that resemble rules. Optimize long enough and you will always find one.

AI tools make this faster and harder to catch. Where it once took patience to iterate through parameter combinations, a modern AI-assisted backtester can cycle through hundreds of configurations in minutes and surface the best-looking result with no disclaimer. You see a clean equity curve. You don't see the configurations that were quietly discarded to produce it.

The Clearest Tell: High Win Rate, Near-Zero Real Edge

Win rate is the metric curve-fitting inflates most reliably. It's easy to optimize and easy to screenshot. A strategy showing 70–74% wins sounds compelling. But across our 660,005 backtests covering 903 assets and four timeframes (1-Hour, 4-Hour, Daily, Weekly), that range is nearly diagnostic of over-optimization.

The data makes this concrete. Murrey Math Lines posted a 74.3% median win rate — but only 11% of the assets it was tested on actually beat a simple buy-and-hold. Holy Grail Confluence: 73.3% wins, 8% beat buy-and-hold. RSI Mean-Reversion: 71.7% wins, 10% beat. SMC: Liquidity Sweep: 71.2% wins, 8% beat. High wins, near-zero real edge — that pattern repeats.

When a strategy wins often but still can't beat a passive position, it usually means the exits are tuned to harvest small gains while letting losses run, or the optimization window happened to be a sustained trend where almost anything wins. Either way, the win rate is doing marketing work, not analytical work.

More Parameters, More Ways to Fit the Past

Every additional input — an RSI length, a multiplier, a filter threshold — gives the optimizer more room to conform to historical noise. A two-parameter strategy has limited flexibility. An eight-parameter strategy can be shaped into almost any historical equity curve.

Ask: how many inputs does this strategy have, and was each one chosen for a stated theoretical reason or did it emerge from an optimization run? If the creator can't explain why the RSI length is 14 rather than 11, that's a flag. If the answer is 'I tested 50 values and this one worked best,' that's in-sample tuning, not research.

This is also why no Smart Money Concepts strategy in our dataset beat buy-and-hold across the assets we tested. The setups have many discretionary conditions and are often described in terms that can be read into almost any chart after the fact — which is another form of fitting the past.

What a Real Out-of-Sample Test Looks Like

A genuine out-of-sample test evaluates a strategy on data it was never exposed to during parameter selection. The discipline: optimize on one period, evaluate on a later period you haven't touched, and don't adjust parameters after seeing the result. The moment you revise the strategy to improve the holdout result, that holdout period becomes in-sample too.

Across our 660,005 backtests we applied one fixed methodology: each indicator ran in its standard or most common configuration against every applicable asset in every tested timeframe, with realistic transaction costs, and without any per-asset parameter tuning. Parameters were not adjusted after seeing the results. That constraint is what makes the output meaningful rather than decorative.

The result is also genuinely humbling: only 63% of the 903 assets we tested had any indicator beat buy-and-hold. Only 26% of all indicator-asset combinations beat buy-and-hold. Honest out-of-sample tests almost always return less exciting numbers than in-sample ones. If a published backtest shows near-perfect results across many assets and timeframes, that is not evidence of skill — it is a sign the parameters were selected after seeing the outcome.

A Note on What These Results Are

Every result on this site is a hypothetical backtest run on historical price data. Past performance does not predict future results. Markets change, liquidity changes, and an indicator that had edge over one period may lose it in another. Nothing here is a recommendation to buy or sell any security. You should not rely on these results as the basis for investment decisions.

That said, hypothetical testing done honestly — broad assets, fixed parameters, realistic costs, no post-hoc selection — is more informative than an anecdote or a promotional equity curve. The methodology matters more than the numbers themselves. The questions worth asking of any backtest are: when were the parameters chosen, on what data, and has anyone tried to break the result?