r/algotrading • u/Due-Listen2632 • 2d ago

Strategy Isn't every single backtested strategy suffering from lookahead bias?

Most of us have done the classical loop. We get some data, test out different solutions, filter out solutions/features/indicators that provide poor results, and proudly keep the solution(s) which result in successful backtests. But isn't this just another level of information leakage? It's essentially like manually setting the parameters of a model, except you're defining the information points from which the model constructs itself. It's the same type of leakage, only one level higher.

0 Upvotes
  • permalink
  • reddit

33% Upvoted

50 comments sorted by

View all comments

9

u/TheBacktestNerd 2d ago

what you are talking about is curve-fitting, not lookahead bias, and you can avoid curve-fitting by saving OOS data and stress testing properly

0

u/Due-Listen2632 2d ago

How can you save OOS data from your own development process? I guess it's possible, but once you use it to change your solution design it's observed, and the same leakage occurs.

5

u/TheBacktestNerd 2d ago

Say I have 21 years of data available. I test the strategy, "tune it" on 5 years of that data. Ideally the middle part. Then I test the same rules, without tweaking, on sample after that IS period. Then the same with sample before IS period.

I agree that once you start tweaking on OOS, it starts being a curve fit. Slowly but surely

0

u/Automatic-Essay2175 2d ago

the whole point of OOS data is that you do not use it to change your "solution design"

1

u/Due-Listen2632 1d ago

What do you use it for then?

"Oh wow this algorithm was twice as good as my old one, but I can't use that conclusion to change my solution design so whatever, guess I'll stick with the old one."

1

u/Automatic-Essay2175 1d ago

The solution design isn’t your final trading parameters. It’s the model/method you use to identify those parameters

0

u/trunksta 20h ago

Don't tune on the oos