Everyone checks the obvious leak. You make sure your signal doesn't trade today's close using today's close. Done, tick the box, move on.
Then the backtest still looks too good, and you can't figure out why. In my experience the leak that survives isn't in the signal. It's in the plumbing around it.
Warm-up windows
Take a 200-day moving average. For the first 199 days of your data, it either doesn't exist or it's computed on a shorter window, depending on your library. Some tools quietly back-fill it. Others use whatever data is there and call it a 200-day average.
Neither is wrong by itself. But if your rule fires on day 40 using a "200-day" value built from 40 bars, you've tested a different strategy than the one you'll run live. Worse, if the values were back-filled from later data, you've peeked.
The fix is boring. Throw away everything before the longest window has fully filled. Then check that your first trade date is where you think it is.
Here's a quick sanity check. Print the first five dates your system actually traded each coin. If any of them falls within the first month or two of the data, be suspicious.
The universe you chose today
Here's the one that bites crypto traders hardest. You pick the top 20 coins by market cap, then backtest back to 2019. But you picked those 20 today.
Coins that collapsed, got delisted or quietly died aren't in your list. You tested on the survivors. Of course trend following looks great on assets that went on to become big.
A cleaner way: at every rebalance date, build the universe using only what was knowable on that date. Rank by volume or market cap as of then. It's more work, and the result is almost always worse. That's the honest number.
Picture a made-up example. A test shows 60% a year across 20 coins. Rebuild the universe date by date and it drops to 25%, because half the winners weren't in the top 20 yet. Nothing about the signal changed. Only the honesty of the test did.
Benchmarks and other helpers
Normalizing by a benchmark sounds harmless. But if you scale your signal by the full-period average volatility, or z-score it against the whole sample, you've handed your 2020 trades knowledge of 2024.
The same goes for any parameter you fit on the full dataset and then apply inside it. Thresholds, volatility targets, even "average spread." If it was computed with data from after the trade, it's a leak.
How to hunt for them
Here's a test I like. Truncate your data at some date, say the end of 2022. Run the whole pipeline and record every trade up to that date. Now rerun on the full dataset and compare the same period.
If any trade changed, something is reaching forward. Same entries, same exits, same sizes, or you have a leak. It takes ten minutes and catches things code review never does.
A second check: shift every signal one extra bar later. A real edge degrades a little. A leak-driven one often collapses, because it was relying on information it shouldn't have had.
What to do with this
Pick your best-looking backtest and run the truncation test tonight. Don't tweak anything first. Just see whether the early trades survive.
If they don't, you've found the reason the live account disagrees with the chart. That's a good night's work, even if it kills a favorite.
It's the sort of check I want on every system before it touches real money. We lay out how we validate results at v33systematic.com.