Search enough rules on pure noise and one of them will look brilliant

Brock, Lakonishok & LeBaron (1992) found simple moving-average and breakout rules genuinely predictive on 90 years of Dow data. Correcting the same rule universe for data snooping erases most of that significance (Sullivan, Timmermann & White 1999). This rebuilds the reason, live, on synthetic data: search a universe of trend rules, keep the winner, then ask the honest question — how well would the best of N rules have done if none of them had any edge at all?

The rule universe you search

Universe size--

Every rule is scored, the best one is kept. Nothing else about the search changes.

The synthetic sample

Sample length--

Daily bars. The first 200 are used only to warm the slowest average up, so every rule is scored on the identical window.

True edge injected0.00

0 = a pure random walk, in which no rule in the universe has any real predictive power. Above 0 adds a slow unobserved drift a trend rule genuinely can detect.

One-way cost10 bps

Sample#1

Same generator, new random draw. Watch how much the winning rule moves with luck alone.

The bootstrap

Replicates150

Each replicate reshuffles the return series — destroying any predictability while keeping the return distribution — then re-runs the whole universe on it.

Runs in your browser in chunks, so the page stays responsive.

Snooping-adjusted p-value of the winning rule

where the winner falls in the null distribution of the best rule in the universe

Naive p-value, winner alone

as if you had only ever tested this one rule

Gap the search opened up

adjusted minus naive

Winner’s score

annualised Sharpe, net of cost

Expected best score by luck alone

mean of the best-of-N null

Rules evaluated to find this winner--

Every rule’s score · the winner marked

Each rule searched Best rule kept Expected best by luck (after bootstrap)

Winning rule · hypothetical index, net of cost (100 = start of window)

Winning rule Buy & hold the synthetic series

Two null distributions of the annualised Sharpe · one rule vs the best of all of them

Null for the winning rule alone Null for the best rule of the whole universe Score actually observed

What you are looking at: with the injected edge at 0 the series is a pure random walk, so every rule’s real predictive power is exactly zero — and yet the best of N still looks excellent, and looks better the larger N gets, because the maximum of many noisy statistics is itself large. That is why the amber distribution sits to the right of the cyan one: luck alone hands the best rule a score no single rule should expect. The naive p-value will often call the winner significant; the snooping-adjusted p-value usually will not. Raise the universe size and watch the naive p-value collapse toward zero while the adjusted one barely moves — that widening gap is the whole of the problem. Then raise the injected edge: both p-values shrink together, which is how you know the correction is not simply destroying everything — it destroys findings that were only ever the best of a long search.

What the research actually found

Support level for moving averages & trend rules: split (mixed). The literature does not agree, and IndicatorEdge does not pretend it does. “The modern debate starts with Brock, Lakonishok & LeBaron (1992), who found simple MA and breakout rules genuinely predictive on 90 years of Dow data.”

“On 1897–1986 data, MA rules produced buy signals that reliably preceded higher returns than sell signals (Brock et al. 1992) — the single most-cited pro-TA result in finance.…”

“The edge largely dies out of sample. Correcting the same rule universe for data snooping erases most significance (Sullivan, Timmermann & White 1999); … and rules that predicted small-cap and NASDAQ indexes stop working after ETFs made those indexes cheap to arbitrage (Hsu, Hsu & Kuan 2010). Nobody has shown a simple public MA rule beating costs in modern large-cap equities.”

The survey literature (Park & Irwin 2007) counts a majority of studies finding positive gross returns, with the honest caveats that costs and snooping cut deep.

What this page does and does not claim. It implements no named test, statistic or number from any of these papers. It demonstrates the general principle a snooping correction rests on: bootstrap the distribution of the best rule’s performance across the entire universe you searched, rather than the distribution of one rule’s performance, and judge the winner against that. Everything on this page is computed live from a seeded pseudo-random generator on synthetic data.

Brock, William, Lakonishok, Josef, & LeBaron, Blake (1992). “Simple Technical Trading Rules and the Stochastic Properties of Stock Returns.” Journal of Finance, 47(5), 1731–1764.

Sullivan, Ryan, Timmermann, Allan, & White, Halbert (1999). “Data-Snooping, Technical Trading Rule Performance, and the Bootstrap.” Journal of Finance, 54(5), 1647–1691.

Park, Cheol-Ho, & Irwin, Scott H. (2007). “What Do We Know About the Profitability of Technical Analysis?” Journal of Economic Surveys, 21(4), 786–826.

Hsu, Po-Hsuan, Hsu, Yu-Chin, & Kuan, Chung-Ming (2010). “Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias.” Journal of Empirical Finance, 17(3), 471–484.