Data snooping: why the best backtest of 480 is worthless
Search a universe of technical rules on a random walk with zero predictability, find the winner, and watch it look excellent. Then bootstrap the distribution of the BEST rule rather than of one rule, and read the two p-values side by side.
Self-contained simulation on randomly generated synthetic data — not market data, not a live signal, and not a record of trading. It runs entirely in your browser; nothing is sent anywhere. Educational information only — not investment advice. Hypothetical backtested results; past performance does not guarantee future results. Trading involves risk of loss. Full-page version, with the findings and citations
What the research actually found
The support rating below is the same one this concept carries in our research corpus, and the quoted findings are the corpus text — not a summary written to flatter the simulation. Every citation was verified against its source.
Moving averages & trend rules (MA cross, breakout filters)
also called: golden cross, death cross, trading-range breakout, MACD as trend filter
Where it comes from. The oldest tested family. Charles Dow's editorials (1900s) informalized trend; the first rigorous academic test was Fama & Blume (1966) on filter rules, which found nothing after costs. The modern debate starts with Brock, Lakonishok & LeBaron (1992), who found simple MA and breakout rules genuinely predictive on 90 years of Dow data.
What the research supports. On 1897–1986 data, MA rules produced buy signals that reliably preceded higher returns than sell signals (Brock et al. 1992) — the single most-cited pro-TA result in finance. Technical indicators also carry real forecasting information for the aggregate equity risk premium, especially around business-cycle peaks (Neely, Rapach, Tu & Zhou 2014).
What it does not support. The edge largely dies out of sample. Correcting the same rule universe for data snooping erases most significance (Sullivan, Timmermann & White 1999); measured trading costs exceed the break-even costs of the BLL rules (Bessembinder & Chan 1998); and rules that predicted small-cap and NASDAQ indexes stop working after ETFs made those indexes cheap to arbitrage (Hsu, Hsu & Kuan 2010). Nobody has shown a simple public MA rule beating costs in modern large-cap equities.
Sources: Fama & Blume (1966) · Brock, Lakonishok & LeBaron (1992) · Sullivan, Timmermann & White (1999) · Bessembinder & Chan (1998) · Hsu, Hsu & Kuan (2010) · Neely, Rapach, Tu & Zhou (2014) · full concept-by-concept evidence
6 sources — all verified
- Fama, Eugene F., & Blume, Marshall E. (1966). “Filter Rules and Stock-Market Trading.” Journal of Business, 39(1), 226–241.
- Brock, William, Lakonishok, Josef, & LeBaron, Blake (1992). “Simple Technical Trading Rules and the Stochastic Properties of Stock Returns.” Journal of Finance, 47(5), 1731–1764.
- Sullivan, Ryan, Timmermann, Allan, & White, Halbert (1999). “Data-Snooping, Technical Trading Rule Performance, and the Bootstrap.” Journal of Finance, 54(5), 1647–1691.
- Bessembinder, Hendrik, & Chan, Kalok (1998). “Market Efficiency and the Returns to Technical Analysis.” Financial Management, 27(2), 5–17.
- Hsu, Po-Hsuan, Hsu, Yu-Chin, & Kuan, Chung-Ming (2010). “Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias.” Journal of Empirical Finance, 17(3), 471–484.
- Neely, Christopher J., Rapach, David E., Tu, Jun, & Zhou, Guofu (2014). “Forecasting the Equity Risk Premium: The Role of Technical Indicators.” Management Science, 60(7), 1772–1791.
This simulation demonstrates a mechanism on synthetic data; it does not reproduce any paper's dataset or reported magnitudes. Educational information only — not investment advice. Educational information only — not investment advice. Hypothetical backtested results; past performance does not guarantee future results. Trading involves risk of loss. See the methodology and the full disclaimer.
Get the weekly edge report
The best-performing indicator per asset, what changed this week, and the honest caveats — straight to your inbox.