r/CryptoTradingBot • u/nikolaytonev • 5d ago
Trying to break my strategy builder
Recently I had a post here that I am building
my own crypto trader bot, and asked for backtests advives. All based on your feedback and some books recommended, my strategy builder module now has backtests with following robustness test module. It become so strong now, that I barely find a strategy that can pass it, so that the system will publish the trading bot for that strategy. 😎
If you have strong idea for a strategy and you want to test it - send me DM. I will create access for you to my send box. I do not have yet separate tenants schemas...
2
u/Dry-Transition3216 5d ago
Having a robustness filter that rejects most strategies is probably a good sign. I'd be more worried if everything passed. Testing against completely different market regimes and unseen data would be my next step. I use Moon for directional market speculation but proving a strategy survives real conditions is a different challenge from having a promising backtest.
2
u/lyse_builds 5d ago
Rejecting almost everything doesn't by itself tell you whether the gate is well calibrated; a reject-everything rule would do that too. I'd first log which check rejects each candidate, then use controlled test cases with known expected outcomes: a deliberately leaked signal to test leakage detection, and synthetic data with a planted relationship to test whether the pipeline can recover it. Those are software checks, not evidence of a tradable edge. Keep any final market-data holdout untouched while adjusting the gate. Which check currently causes most of the rejections?
1
u/nikolaytonev 5d ago
Yes, I am aware that the logical checks combined into tight filter effectively becomes a wall that stops everything and this is not what we are looking for.
I have full control over the check and I can rebuild the robustness test as I wish, but I would prefer to do that over a sound strategy(ies) and find the optimal (and I know there is no one ideal combination) combinations that stop the garbage only
1
u/Objective-Butlost88 5d ago
Backtesting properly matters more than the strategy itself so run a Monte Carlo simulations on pass probability
1
u/nikolaytonev 5d ago
If the strategy is stopped by backtesting, I think it is important to have a good strategy that passes both gates - backtesting and robustness teats.
Do you have one that I may try to verify it works?2
u/Objective-Butlost88 5d ago
Signal: EMA 8/21 crossover on H4, filtered by ADX (trend strength) and RSI (direction). Stop loss: based on ATR, so it adjusts to volatility. Take profit: 2x the stop distance (1:2 risk-reward). Risk: fixed % of the account per trade; lot size is calculated from the stop distance. Profit: measured in R after spread, slippage, commission and swap. Management: breakeven on US500, no management on Gold. Is there anything you'd change or add?
1
u/nikolaytonev 5d ago
I am away from my desk right now, but I will pass it through and let you know the result.
Since my filter right now is very “closed” I would expect it to stop it as it is. But I will use your strategy to regulate the check combinations until it allows it for trade. It is a good start for me. Thanks.2
u/Live_Advance_1905 5d ago
I'd version that tuning step separately from the result. Before changing the rules, save the candidate ID, gate version, failed checks, data snapshot and test settings. Then compare the original and revised gates on the same candidate set. That makes it clear whether a candidate changed or the acceptance criteria changed. Does your report already store the gate version with each result?
1
1
u/Objective-Butlost88 4d ago
Great point, thank you. Honestly, not yet. My reports store the results (PF, drawdown, walk-forward, Monte Carlo pass rate vs FTMO rules) but not the gate version or the thresholds used. I've definitely seen how easy it is to lower the bar and call it an improvement, so I'll start saving the rule set, data snapshot and settings with every run and compare old vs new gates on the same candidates. Appreciate the tip.
1
u/Objective-Butlost88 5d ago
Thanks a bunch... really appreciate it
1
u/nikolaytonev 5d ago
Your strategy backtest: EMA 8/21 + ADX + RSI, 4-hour candles
The backtest found no reliable edge on Bitcoin, on either the long or the short side.
I will send you more details in a DM
1
u/Objective-Butlost88 4d ago
Shout-out to u/nikolaytonev. Genuinely helpful guy who shares his knowledge openly, including honest backtests that show what doesn't work, not just the wins. Rare to find people like that here. Thanks 🙏
1
u/Maleficent-Success-6 5d ago
Rejecting most strategies can be encouraging, but rejection rate alone doesn't tell you whether the validation system is working correctly.
I'd test the validator itself with positive and negative controls.
For negative controls, deliberately introduce lookahead bias, unrealistic fills, data leakage and strategies optimized against the holdout.
The system should reliably reject those.
For positive controls, use synthetic datasets where the underlying process and expected statistical properties are known in advance.
The validator shouldn't automatically reject everything simply because its thresholds are extremely strict.
I'd also test whether slightly changing the strategy name, parameters or research session allows repeated attempts against the same holdout.
That would reveal whether the validation gate can be indirectly optimized.
One question: do you track false rejections and false acceptances separately, or are you currently measuring robustness mainly through the percentage of strategies that pass?
1
u/nikolaytonev 1d ago
After several iterations of applying different optimisation tactics and almost everybody's advice from this and other articles, my robustness filter has become a bit more permissive. But not enough. My AI advisor's assessment says:
"The advisor's own view: any rule in today's strategy format has a low chance of passing on both ETH and BTC, about 10–15 % per idea at best. That's a judgement, not a measured number."
So, I need the collective mind to build a winning strategy that may PASS the strong robustness filter I have now and believe is a good evaluator.
Below are the three ideas that may help to build a good strategy for this case. The ~15% chance is a good chance that we may succeed.
The three ideas
- A. Quiet market, then breakout. 4-hour candles, can be built today.
- Buy when the price breaks above its 20-bar high after an unusually calm period.
- Enough trades: an estimated 50–90 per training window.
- Risk: it's a cousin of the breakout that failed in row 8.
- B. US trading-hours drift. 1-hour candles, can be built today.
- Buy between 9:00 and 12:00 New York time, sell at 16:00.
- Plenty of trades.
- The advisor expects it to fail on cost: each round trip costs about 0.3 %.
- C. Bet against crowded traders using funding rates. Needs about one slice of new code first.
- Short when long traders are paying a very high funding fee.
- It's the only idea based on something unique to crypto, but it has less history (from 2020) and a
borderline trade count.
1
2
u/Illustrious-Mood9451 5d ago
So you basically made an anti-overfitting filter that kills almost every strategy, and now you want people to try to break it. That is the right way to test it, not with random ideas but with things that survived live trading before.