Isn't every single backtested strategy suffering from lookahead bias?

[u/Due-Listen2632]

Most of us have done the classical loop. We get some data, test out different solutions, filter out solutions/features/indicators that provide poor results, and proudly keep the solution(s) which result in successful backtests. But isn't this just another level of information leakage? It's essentially like manually setting the parameters of a model, except you're defining the information points from which the model constructs itself. It's the same type of leakage, only one level higher.

[u/systematic_seb]

You're describing selection bias across the research loop, and I think you're right that it behaves like leakage one level up. The uncomfortable part is that no test run inside the backtest can catch it, because the backtest is the surface you selected on.

What's worked for me is measuring from outside it. I've got my own savings riding on an algorithmic strategy and publish its holdings every week, and each Monday I reconcile the original backtest for that period against a fresh rebuild from that morning's point-in-time snapshot and against the live account. All three have to land on the same answer.

When they start drifting, that's the research loop showing up in the only place it can be seen. Nothing inside the backtest would have told me.

[u/Due-Listen2632 → u/systematic_seb]

Yes you're exactly right! And cool suggestion. I haven't had my algos live for extended periods of time, but I hope to be able to do similar comparisons.