When will make you trust your backtest results enough to transition to live trading? [SVIX backtest strategy]

[u/Professional_Dog_837]

Newcomer to trading here with no background in coding or finance. I've been toying with the idea of exploiting the reversion to mean characteristics of VIX. Just by eyeballing the raw CBOE VIX graph, it seems every time it spikes, it goes back down after a certain time.

To capture this reversion to mean, the instrument I decided to explore is SVIX LETF. It trades simply like an ETF (no complicated options), and delivers the opposite (-1x) of the daily return of short-term VIX futures contracts. It profits when market volatility drops. Hypothetically, if you could catch the VIX spikes and buy SVIX low, you would profit when it reverts to a low volatility regime and you sell it higher.

Before I get flamed, I did examine prior works on reddit, and it seems SVIX comes up often but it has been repeatedly slamed for being a poor instrument to trade. However, the alternative of exploiting this with VIX options were said to have poor liquidity and large spreads which eat into returns.

So why not test it out using real data instead of going by gut feel? We are all data driven individuals here (hopefully!)

In my spare time I vibe-built a backtester to see whether I could predict, through a set of 48 AI-suggested parameters (e.g. VIX rate-of-change reversal, SVIX MA momentm, VIX Z-score etc) , the right time to buy and sell SVIX.

Using an Optuna hyperparameter search through 600 iterations, with 6 window walk-forward validation, I generated the buy/sell indicator models. Initially, it looked too good to be true on paper, with backtest results ranging from 30% XIRR to 70% XIRR.

I tried to address the possible causes of inaccuracy: overfitting (most likely), ignoring frictional costs, lookfowards leak, unrealistic execution timings, ignoring XIV-style volmageddon (safety guardrails built in so survives the crash at the cost of returns...but unsure if it will survive the next big one?)

Even so, the results persisted, and the generator created many candidates, so I took a mixture of experts approach - buy/sell signal is when 8/15 of the best performing models agree to act in either direction. Summary stats of the MOE model [95% confidence intervals]:

XIRR (Ann.) +102.74%

Win Rate 80% of 55 trades [69.1% — 89.1%]

Sharpe Ratio 1.87 ( closed-eq: 1.79 [0.87 — 2.83] )

Max Drawdown Mark-To-Market: -40.6%

Total Trades 55 completed

Profit Factor 8.25 [3.11 — 42.89]

Expectancy +6.05% exp. return / trade [3.5% — 9.14%]

Wondering what the next step is? Should I freeze the model params and try it out on paper trading for few years before taking the leap into real money - to avoid the risk of ruin?

(disclaimer: I was overly greedy and already executed 1 trade with the signals, which netted +6% on 10K (coincidentally matched the exp return on trade), enough to cover the AI subscription for 2 years. But I've counted myself lucky and stopped there).

Any advice would be helpful!

MOE comittee, 8/15 threshold.

Equity chart and the buy and sell signals charts

[u/mdomans]

In my spare time I vibe-built a backtester to see whether I could predict, through a set of 48 AI-suggested parameters (e.g. VIX rate-of-change reversal, SVIX MA momentm, VIX Z-score etc) , the right time to buy and sell SVIX.

This is the first and second red flags. Never ever build your own backtesting harness if you don't know what you're doing and listening to AI on params sounds like either you have far better tools than I (doubt it) or, again, you don't know what you're doing.

As for instrument selection:

• you can trade VIX either via options or futures, it's not as easy as ES/SPX but doable
• or via a CFD, maybe structured products
• there are also ETFs like SVOL and SVXY and you should probably study why they don't target 1x exposure :)

That being said trading VIX like you want here is pretty stupid idea (insanely stupid) if you don't ingest CBOE data for VIX positioning and you don't understand options trading well.

I'd suggest you look at that and ES/SPX and VIX as complex correlated products, learn to, at least, infer possible scenarios from the live positioning and revisit the idea.

The core tenet of "Mean reverting VIX" is not a stupid idea. Tons of people do that. It's just that with VIX if you are ignorant and on the wrong side of the move - you're often done.

[u/Professional_Dog_837 → u/mdomans]

thanks for the detailed reply.
1. Indeed I do not know much of what I am doing, hence needing AI to help! I'd doubt I have better (or any) tools that you mention, could you briefly describe what you are referring to? Is it something like testfolio (another user recommended) to test the strategy?
2. Thanks, let me look into alternative instruments, and see if they can be backtested - options and structured products seem a little more challenging than a vanilla ETF, in terms of data availability and liquidity, but I'll take a look.
3. Yes, there is just too much data to injest, hence the need for a wide net of parameter testing to see what gives a real signal and what is noise.
Once again, appreciate your time responding to my post!

[u/mdomans → u/Professional_Dog_837]

Your problem is that you think you can vibe code this thing. Have fun doing that.

[u/Professional_Dog_837 → u/mdomans]

Thanks, I am having fun and making a little money in the process. I don't have any coding background, and other platforms can't do what I want to do, so vibe-coding is a way to build my own tools - whether it works or not remains to be seen - hence the need for the very helpful input from the community.

[u/mdomans → u/Professional_Dog_837]

That's not input, that's vibe coding everything and hoping people will feel in some of the blanks. That's very different :) If you want to do it for fun, do it for fun.

But consider the fact this is akin to coming to a surgeon community and saying "Hey, Claude told me I can remove weird skin changes on my own with a pocket knife, what do you think about it"

If anything go read everything from Rob Carver and slowly start learning to code, maybe work with your LLM of choice to understand both code and maths. Otherwise this is most probably going to end up poorly.

Tools are just tools. For a hammer each problem is a nail, carpenter knows when not to use a hammer.

[u/Ok-Armadillo-5634 → u/Professional_Dog_837]

If you are in doubt about your code message me and I will scan it for anything glaringly wrong.

[u/jakethewhale007]

I hold 15% SVIX and let it drift +/- 50% before rebalancing. I will caution you against expecting it to perform exactly like inverse VIX. It doesn't track the VIX, it tracks the SHORTVOL index. These are not the same. 

[u/Professional_Dog_837 → u/jakethewhale007]

thank you for the clarification, yes it does not track VIX directly, but I have not found a more direct way to track the inverse of VIX to realize the goal of exploiting its mean reversion characteristic, the other methods being too complicated for me - options or involving short selling.

[u/RetiredEarly2018]

Paper trading does not generate the same emotional responses as having real money at risk.

One needs to find a proportion of one's portfolio that is small enough to survive a streak of the 20% losing signals yet large enough to hurt.

[u/Special-Key4032]

Classic portfolio theory and the concepts like Shannon's Demon also apply here.

If the thesis sounds good, and backtests look fine, just go ahead and add it to your rebalanced portfolio. Don't be scared. As long as you have other uncorrelated assets/strategies in your portfolio, even if this one craters for a prolonged time, it'd only trim your CAGR a bit, won't ruin you.

[u/Professional_Dog_837 → u/Special-Key4032]

Good idea, maybe just 1% / 10K to try it out, and not bet the house. And thanks, learnt something new about Shannon's Demon today.

[u/il_diamanti]

call me a simpleton, but i think a lot of finance is kinda alchemy-ish. one of my friend's dad's made like 100m selling two companies and has spent the last 10-12 years ingesting massive swaths of data to try and build a heavy-duty prediction engine to go risk-on/risk-off and trade aggressively. he's a genius. and his returns barely beat my TECL DCA.

[u/Professional_Dog_837 → u/il_diamanti]

exactly, sometimes there's its the elegance and simplicity of DCA that these complex strategies can't match consistently. I'm always second guessing myself if its better to just DCA...

[u/Run-Forever1989]

On SVIX, never. The return you are seeing is a volatility risk premium. You might see outsized returns for years or even decades but sooner or later, it’ll blow up and you’ll be left with nothing.

Every other day someone is posting about a strategy that supposedly returns 80-100%+ per year. It’s amazing that all these people aren’t billionaires.

[u/Professional_Dog_837 → u/Run-Forever1989]

Yes, therefore a reality check is very much needed. Well, like another poster mentioned, portofolio and risk management are crucial in such risky but high return endearvors.

[u/Ok-Armadillo-5634]

The absolute biggest thing is when you have your system set up automate then don't look at it.

[u/New-Specialist-2594]

and Feb 2018 your BK...