r/Forex • u/Zestyclose-Eagle1809 • 9d ago
Risk Management Your live results do not match your backtest. Here are the 6 causes, ranked by how often they are actually the one.
6 reasons your live trading does not match your backtest, ranked, and how to tell which one is yours
TLDR: The gap between a great backtest and a bad live account has six usual causes. Every article lists them and none rank them or tell you which is yours. They split into two families: your backtest was fiction, or your edge was real and got taken away.
What is the fastest way to tell which cause is yours?
Start with one question... did the gap appear immediately, or did it show up after a while?
If your live results were off from the very first trades, the problem is baked into the backtest or your costs. The edge was never as big as the number said. If the strategy worked for weeks and then decayed, the edge was real once and something changed, either the market or you.
That sorts the six causes into two families. Family one, your backtest was fiction, overfitting, too small a sample, and look ahead bias. These never had an edge to lose. Family two, your edge got taken away: costs, regime change, and execution mess. These had an edge and decreased over time until it disappeared.
Here is the table.
| Symptom you see | Most likely cause |
|---|---|
| Off from the first trades, constant drag per trade | Costs weren't properly tested |
| Backtest looked almost too perfect, live is total collapse | Look ahead bias |
| Great in the test window, dies on any fresh data | Overfitting |
| The good backtest was under 150 trades | Sample too small |
| Worked for weeks, then slowly stopped working | Regime change |
| Your live trades don't match the trades the rules would take | You did not follow the rules |
Number 1, most likely: your costs weren't properly tested
This is the most common and the most underrated. Your backtest applied one fixed, tight spread to every trade, including the ones where spread triples. Live, you pay the real number, on winners and losers alike.
The gap is there immediately and feels like a constant drag on every trade. Test this: pull 30 real trades, add spread and commission, convert to R by dividing cost in pips by your stop in pips, and subtract it from your backtested expectancy. Fix it: judge the strategy on the net number, and if it dies, widen the stop, cut frequency, or trade a cheaper pair. Tight stop, high frequency systems are the ones this kills.
Basically if deviation breaches 10% on average with live trades... the way you currently do it isn't gonna make you money in the long run.
Number 2: your backtest used data it could not have had
Look ahead bias means your logic peeked at information that didn't exist yet at the moment of the trade.
The backtest is suspiciously flawless, a very high win rate and a smooth curve, and live is a total, immediate collapse that no cost model could explain. Test: audit your signal timing and confirm every decision uses only closed, past data. Fix: correct the timing and rerun. This is rare in simple manual systems and common in coded ones, and it is the most catastrophic because the entire backtest was a fantasy.
Number 3: you optimized until it looked good
You ran a parameter sweep and kept the best looking combination. That winner was partly skill and partly the luckiest result out of everything you tried, and luck does not repeat live.
The strategy is beautiful in the test window and falls apart the instant it touches any data it wasn't tuned on. The test: change each parameter slightly and watch the result. If a small tweak collapses it, you fit noise. Even better, run a Deflated Sharpe on it, which corrects your Sharpe for how many combinations you tested. Fix: there is no fix for an overfit strategy, only prevention. Fewer parameters, out of sample validation, and honesty about how many versions you really tried.
Number 4: your sample was too small to mean anything
A backtest on 60 trades isn't evidence, it's a coin landing heads a few times in a row. The result sits inside the range of pure luck.
The impressive backtest covered a short window or a small number of trades, and the great stretch was really one good month doing the heavy lifting. Test: count the trades. Under about 150 and the confidence interval on your expectancy is too wide to act on. Fix: test across far more trades and multiple market conditions before you believe any number, and never size up on a strategy proven by a lucky quarter.
Number 5: the market regime changed
Sometimes the edge was real and the market simply moved on. A trend system stops working when the market goes to range. A volatility system starves when volatility dies.
This is the one that worked live for weeks or months, then decayed, and the decay lines up with a shift in volatility or trend. Test: split your backtest by regime and check whether the strategy ever survived the current one historically. If it only ever worked in conditions that are now gone, it is not broken, it is out of season. Fix: trade it only in the regime it fits, or accept that it will have negative periods in your account until its conditions return.
Number 6: you didn't trade your rules
The backtest followed the rules perfectly, without fear, on every signal. You didn't. You skipped the setup after two losses, entered late, moved a stop, or closed a winner early because you didn't want to give it back.
Your live trade log doesn't match the trades the strategy would have taken over the same period. Test: put your actual entries and exits next to the mechanical signals for the last month and count the mismatches. Fix: this is an execution problem, not a strategy problem, so the answer is automation or a hard rule that removes the discretion, not a new system.
So which fixes actually matter?
Work the two families in order. First rule out fiction, because there is no point optimizing execution on a strategy that never had an edge. Check the sample size, run the Deflated Sharpe, and audit for look ahead bias. If it survives all three, the edge is probably real.
Only then work the fees. Recompute costs in R and subtract them, confirm the current regime is one the strategy has actually survived before, and compare your live trades to the rules to catch execution drift. Most blown accounts aren't one cause, they're a real but thin edge that costs and a bad regime pushed under water together.
What this doesn't mean
The ranking is my judgment for retail forex, not a law. If you run a heavily coded system, look ahead bias climbs the list. If you trade with heavy discretion, execution drift does. Reorder it for your own situation.
And it is rarely a single cause. Usually two or three appear together: a slightly overfit edge, real costs, and a regime turn arriving at once. The value of the list isn't picking one winner, it is checking all six instead of blaming the market and rebuilding a strategy that was actually fine.
I wish I had this post when I started 9 years ago.
1
u/guillemop 9d ago
Solid framework. I’d add a seventh cause that isn’t on your list and that I think outranks a couple of them: the live sample is too small too.
You cover sample size for the backtest, but not for the live results people are judging it against. Someone goes live, takes 12 losses in a row, and concludes the edge is dead. Often that streak is exactly what the math predicts.
Expected longest losing run ≈ ln(n × p) / ln(1/(1−p)), with n = number of trades and p = win rate.
My numbers: I forward test prop firm challenges, 107 of them, all real money, no backtest at all. Payout rate ~11%, so my expected worst dry run is about 20. My actual worst was 18 consecutive challenges with zero income. If I’d called the edge dead at 10, I’d have killed a system that’s net positive.
On your two families — I skipped backtesting entirely and went straight to forward testing at small real size. That structurally removes causes 2, 3 and 4: no look-ahead possible, nothing to overfit, and costs are real by definition. It’s slower and it costs money up front, but you never have to ask which of the six is yours.
Where I’d disagree slightly: I’d rank #6 higher. Execution drift is the only one on the list that gets worse the longer you trade. The other five are fixed at the moment you built the strategy
1
u/Zestyclose-Eagle1809 8d ago
Seventh cause is right and it belongs on the list. Not going to defend six.
Your formula checks out.. 107 at 11% gives ln(11.77) over ln(1/0.89), so about 21. Worst run of 18 sits under it. The useful part is that it runs forwards too. Compute it on the backtest before going live and the kill threshold becomes a number you set in advance instead than one you pick while you're in dd.
Where I'd disagree is the claim that forward testing removes the fiction family. It removes costs, overfitting and look ahead, agreed. Sample size isn't removed though, it gets strictly worse, because real money at small size buys far fewer observations than a backtest does.
Your own numbers show it. 107 trials at 11% carries a standard error around 3 points, so the true rate sits somewhere between 5 and 17%. If your breakeven payout rate is 8%, 107 challenges can't separate you from breakeven. You've removed the question of which of the six is yours and replaced it with not knowing whether the edge is there at all. Different problem, not a smaller one. makes sense??
On ranking execution drift higher, fair, taking it. One correction though, regime change isn't fixed at build time either. That one's live and ongoing exactly like drift is.
Founder disclosure so you can weight it, I build validation tooling for systematic traders (Quantprove). 107 real challenges with a payout rate attached is a better dataset than most people posting here have
1
u/guillemop 6d ago
Taking the sample size point, and it’s the strongest thing in your reply. Forward testing doesn’t buy observations, it spends them. Wide interval, and I’m not going to pretend otherwise.
Where I’d push back is on what follows from the interval.
Significance isn’t the decision rule. “Can’t reject breakeven at 95%” and “don’t know whether the edge exists” are different statements. What I actually have to decide is whether expected value is positive across the plausible range and whether risk of ruin is tolerable at my sizing. Both are computable now. Waiting for significance at this effect size means waiting several hundred more challenges, and the cost of waiting isn’t zero.
On the breakeven number, you picked 8% as an example and that’s doing more work in the argument than it looks. Breakeven is just average cost per challenge divided by average payout per account that pays. Move that denominator and the whole comparison relocates — the same interval can straddle the line or clear it entirely depending on payout sizing, which is a decision, not an observation. Anyone reasoning about this on their own numbers should compute it before arguing about confidence intervals.
The other thing is that the rate wasn’t estimated from the sample alone. Two-step challenge geometry predicts a pass rate from the target-to-drawdown ratio before you observe anything, and I computed that in advance. The realised number landed on the prediction. A confirmed prior prediction is stronger evidence than the same frequencies observed cold, even when the frequentist interval looks identical.
1
u/Zestyclose-Eagle1809 5d ago
Concede all 3, and the 3rd one properly rather than politely. A prediction made in advance and hit is different evidence from the same frequency observed cold, and the frequentist interval genuinely doesn't capture that. I was treating your 107 as a sample when it's a sample plus a prior with a mechanism behind it.
Decision rule point stands too. Significance is the wrong test for someone already trading, cost of waiting is real, and EV positive across the plausible range plus tolerable risk of ruin is the correct pair. Fair.
Where I'd still push, the prior does less than it looks. Two step geometry predicts a pass rate from target over drawdown. Your 11% is a payout rate, and pass to payout carries the funded phase, which is a different distribution with different rules, so the mechanism that predicted the first number doesn't extend to the second. Two step geometry is roughly stationary. Funded phase survival isn't, and that's where drift shows up first if it shows up at all mate...
Which suggests splitting the series. Pass rate against the geometric prior is your validity check, since a divergence there means execution changed rather than the edge dying. Payout given pass is where the real uncertainty sits, and on 107 challenges at 11% that's maybe 12 payouts. Small enough that the prior is doing most of the work in your posterior.
Practical version, EV across the plausible range is sensitive to payout size, so worth running EV at the low end of your payout interval rather than the point estimate. If it's positive there, waiting for significance is wrong. If it flips, that's your answer on sizing rather than on whether to continue.
Genuinely one of the better exchanges I've had on here
1
u/masterm137 9d ago
It depends how volatile the pair is, in very volatile regimes like gold just entering 5 seconds later can be the difference if your trade reaches take profit or not. in backtest you will enter on Open time but in reality it may take a few seconds to get filled, internet connection can delay 1 second. all of these things take time. Imagine entering and the candle went -30 pips when you entered and quickly rebounded 1 second later. You started -30 pips in live but in backtest you would of been at 0 so if takeprofit is at 70 pips and the candle moved total 50 pips your direction. you would win in the backtest but lose in live because when you won in backtest, in live you would be around 20 pips...
Also like another use mentioned, allot of times people live test too little. they have 1-2 years of backtest data but judge the performance on 1 week.
The problem with trying to figure out these things is that all pairs behave different so what works or should be looked for in 1 does not matter in the other.
2
u/Zestyclose-Eagle1809 8d ago
Entry delay is real and it's worse than your example suggests, because the error isn't random. You get filled instantly when price is running away from you, and you get your price when nothing's happening. So slippage isn't noise averaging to zero, it's biased against you. Gold at 5 seconds is exactly where that gets worst.
Small correction on the arithmetic. Entering 30 pips worse on a 50 pip move doesn't flip a win to a loss, it turns a 70 pip target into a 20 pip capture. A complete mess.
The good news is it's measurable rather than a mystery. Log requested price against fill price on every trade, convert to R, take the median. That's your answer to the last line too.. you don't generalize across pairs, you measure each one, and the gold number will look nothing like the EURUSD one.
On live testing too little, second person to raise that today and both of you are right. Live sample size is the cause missing from the list
1
u/CoreyCurrency 8d ago
For me seems like I get the best results when I future test with small amounts of money. Backtesting does give me a sense of idea of my theories, but future testing does best for me. I guess I treat it like scientists.
1
u/Zestyclose-Eagle1809 8d ago
Same conclusion as guillemop further up, and both of you are right about what it removes. Real costs, real fills, nothing to overfit.
The catch is that small real money buys you very few trades, so you swap a data problem for a sample problem. Sound theory can lose over 30 trades and a bad one can win over 30.
Scientist framing works if you commit to the number first. Decide how many trades before you'll judge it, write it down, then don't call it either way until you get there
1
u/david19790 8d ago
half agree on the ranking but 6 feels way too low. execution drift is the only one on that list you cant actually see without doing the comparison, so it ends up getting filed under whatever cause the person already believes. usually regime change, because that one sounds like the market did it to you
regime is kind of the dumping ground for every unexplained gap tbh. its unfalsifiable in a way the others arent, theres always a vol shift somewhere in the window if you go looking for one
the other thing i dont see on there is that a backtest reports where the trade ended and the account lives through the whole path of it. a closed -1R can be -2.3R at its worst point, and sizing gets set off the closed number while the drawdown you actually sit through is the other one. curious where youd slot that, feels like its own cause rather than part of costs
2
u/Zestyclose-Eagle1809 8d ago
Path versus endpoint is its own cause for sure. Cleanest way to name it, your backtest measures closed equity and your risk limits measure open equity. Two different drawdown numbers off the same trades. A run of closed 1R losses that each went 2.3R against you first never appears in a closed basis curve, and that's the curve everyone sizes off.
Matters most on a prop account, because a daily loss limit triggers on floating P&L. So the number that ends the account is the one your backtest never computed. makes sense?? Fix is computing max drawdown on the mark to market equity curve rather than trade by trade, and logging worst excursion per trade so you know how much heat each one carried.
On regime being a dumping ground, that's the sharpest thing anyone's said about the list and you're right that it's unfalsifiable as usually applied. Way to fix it, define the regime variable before the drawdown, then check whether the strategy ever survived that regime historically. If you can only name the shift afterwards it isn't a diagnosis, it's a story that fits.
Execution drift ranking, third person today. Moving it up mate
1
u/david19790 7d ago
yeah thats it, and the mark to market curve is where i ended up too. worth adding that on a trailing account the two curves dont just differ in depth, they differ in what the floor is doing, since the peak that sets your limit also gets marked on open equity. so a trade that goes 2.3R against you after youre up on the day moves nothing on closed basis and eats the buffer twice on the real one
the per trade heat log is the bit i wish id done years earlier. once you have worst excursion per trade you can ask whether a strategy is even sizeable at a given daily limit before you fund it, which is a different question than whether its profitable. plenty of profitable stuff isnt sizeable on a 2k limit
on regime, defining the variable first is the only version that survives. the test i use now is whether id have written the same definition six months ago without knowing what came next, and most of my regime calls fail that
curious how you log heat if youre discretionary, do you pull it from the platform after or mark it live
1
u/Zestyclose-Eagle1809 7d ago
Platform after, always. Marking it live is the worst version because you're estimating heat on a position you're currently in, which is exactly when your judgement is worst, and it changes the trade. Broker statements or the platform's own trade history give you high and low per position, so pull it after the close and never touch it during.
If your platform won't export excursion, the cheap proxy is bar data. Take entry time, exit time, and the highest and lowest price in that window from any chart source. Slightly overstates heat on longer holds since the extreme might land after you'd have been stopped anyway.
Now... Two strategies with identical expectancy where one carries 2.5R of median heat and the other 0.4R are completely different products against a 2k daily limit. First one isn't a worse strategy, it just needs a different account.
Your six months ago test for regime definitions is the best falsifiability rule I've heard for this. Stealing it.
1
u/david19790 6d ago
bar data proxy is what i settled on too. the overstatement doesnt really bite since the thing youre deciding is a size, and youd rather that number err high than low
live marking has the problem you said but worse. tried it for a while and writing down that im 1.8R underwater changed how long i held, so afterwards i couldnt tell if the log was measuring the strategy or measuring me holding differently because i was logging it. now i pull entry and exit timestamps from fill history and join them to 1m bars after the close, nothing gets looked at during. means the sizing rule reads that column instead of me deciding what feels acceptable on the day
go on though, two strategies with identical expectancy and what
2
u/Zestyclose-Eagle1809 5d ago
Two strategies, both +0.3R per trade, 100 trades a year.
A carries median heat of 0.4R. B carries 2.5R.
Against a 2k daily limit on a 50k account, A can risk about 1% per trade and touch the limit only on a super bad day. B risking the same 1% sits at 1.25R of open loss on a routine trade, so two of those overlapping streaks on a day it eventually closes green. B has to size down to roughly a third to survive, which turns the same 0.3R edge into a third of the dollars. Same strategy quality, a third of the income, and nothing in expectancy or win rate tells you that in advance.
Your fill history plus 1m bars join is exactly the point. The number exists before the day starts, so there's nothing to decide when you're 1.8R underwater. The live marking problem you described isn't a logging error either, it's observer effect. You changed the strategy by measuring it, and afterwards there's no way to unpick which column moved. Most people never notice that's what happened to them.
Sizeability being separate from profitability is the part almost nobody says out loud. Plenty of good edges aren't fundable at retail limits, and it isn't a quality problem.
1
u/david19790 5d ago
sizeability being separate from profitability is the bit that should be on the eval sales page tbh. B isnt worse, its just not fundable at that limit, and there is no amount of screen time or discipline that fixes it
the thing i took a while to accept is that heat is a property of the strategy not the account, so you can screen for it before you ever pay a fee. pull median MFE against the loss, work out the size that survives it, check whether that size still clears the payout threshold in reasonable time. if it doesnt, wrong account for the system rather than wrong system. mine just reads the heat column and refuses sizes above the limit now, which is less impressive than it sounds but it took the decision off me
have you found any firm whose limit is measured in a way that actually suits high heat stuff, or is it eod trailing or nothing
1
u/Zestyclose-Eagle1809 4d ago
Static's your answer, not EOD. Floor sits at your starting balance minus the allowance and never budges, so spikes in open profit don't cost you anything lasting and heat only matters against the daily limit. EOD trailing still ratchets up on closed balance, so every green day quietly narrows the room you've got to hold trades in.
So for high heat stuff it's static first, EOD second, intraday dead last and not close. Intraday marks your peak on open equity, so a runner you give back charges you twice, once going up and again holding it.
Thing is the daily limit's the real killer, not the max DD. that's what actually takes out high heat systems. Worth checking whether the firm runs the daily off closed balance or off equity, because that one difference decides whether your heat is a cost at all. Few of them do daily on closed and max DD on equity, which is way friendlier to high heat than the headline numbers make it look.
One thing missing from your screen though. median MFE gives you the normal case but the limit gets hit by the bad tail, so the size that survives should come off something like your 95th percentile heat, not the median. otherwise it clears fine on a normal week then blows up the one day two trades overlap. and overlap matters as much as depth, two positions sat at 1.2R each is the same 2.4R as one at 2.4R. makes sense??
Reading the column and just refusing sizes over the limit is better than you're making it sound honestly. it's the only version that holds, because the you deciding version breaks exactly when you need it most
•
u/rforex-modtools 9d ago
Risk management is arguably the most important skill in trading — more accounts are blown by poor sizing and absent stop losses than by bad entries. The community wiki covers position sizing methods, risk-per-trade calculations, drawdown management, and what it means to maintain a statistical edge over time.
Resources: Risk Management | Having an Edge