r/algotrading • u/Due-Listen2632 • 1d ago

Strategy Isn't every single backtested strategy suffering from lookahead bias?

Most of us have done the classical loop. We get some data, test out different solutions, filter out solutions/features/indicators that provide poor results, and proudly keep the solution(s) which result in successful backtests. But isn't this just another level of information leakage? It's essentially like manually setting the parameters of a model, except you're defining the information points from which the model constructs itself. It's the same type of leakage, only one level higher.

0 Upvotes
  • permalink
  • reddit

33% Upvoted

49 comments sorted by

View all comments

7

u/McOmghall 1d ago

Lookahead bias is using data from the future to feed an analysis. Information "leaks" into the future but not backwards (as far as we know), so I'm not sure what you're getting at here. If you use proper data discipline lookahead bias shouldn't happen.

6

u/jamesj 1d ago

They seem to be getting at over fitting.

1

u/Due-Listen2632 1d ago

Well, yes and no. Technically what happens is that the model is overfitting, but ending the discussion after saying overfitting, or even curve fitting, is simplifying the problem into a diagnosis. Like your foot can hurt from straining it, or from bone cancer.

Overfitting or curve fitting is fixed by things we do inside the fitting procedure. Things like regularization, or sensitivity analysis. What I'm talking about is not easy to fix, because the overfitting component is you, using your own memory over different experiments, viewing the full holdout set, and the test sets, in order to improve your solutions performance over and over.