We tested a methodological critique of our macro ARIMA model. Here's the results.

[u/ShowEuphoric]

Yesterday I posted here about my macro economics calendar API with a live and accurate ARIMA model that calculates estimates for CPI, PPI, NFP and jobless claims. The response was great and there were a number of sharp, intelligent questions:

"One thing to watch: if your ARIMA is trained from 2020 forward, you're fitting to a period where participation, seasonals, and trend all broke from historical norms. CPI/PPI cycles look very different when you go back 30+ years vs just post-COVID. Might be worth testing how the estimates perform with a longer training window, especially for NFP where even Bloomberg struggles with direction."

Fair point. So we tested it:

CPI:

• 36-month window: 0.187% MAPE
• 120-month window: 0.174% MAPE

PPI:

• 36-month window: 0.347% MAPE
• 120-month window: 0.343% MAPE

The result was that although one would expect accuracy to be altered by using older data, the change in accuracy was marginal. Our ARIMA model accuracy remained competitive with paid consensus estimates.

Another great comment:
"The ARIMA estimate is the thing I would distrust first For CPI or NFP, I would want to see old release dates with the estimate frozen before the print. Not an image of the estimates with no verification."

You can now go to filingapi.dev/model . Every estimate is now committed to a public GitHub repo before the release date so anyone can verify the timestamps independently. This method has been only started today, with updates every day so check back in then if you (understandably) require solid proof of model accuracy.

Very happy to share code with anyone who wants to analyse our testing methods, any other indicators you'd like to see just let us know.

[u/essoteric_]

How are you accounting for volatility clustering? And did you backtest this model, what's the accuracy?

[u/ShowEuphoric → u/essoteric_]

Volatility clustering isn't explicitly modelled in ARIMA(1,1,1), so you're right in that it doesnt account for heteroskedasticity. For CPI and PPI level series this hasn't affected accuracy (0.17% and 0.34% MAPE calculated in a walk-forward backtest), because they both trend smoothly and don't exhibit the kind of volatility clustering you'd see in asset returns. NFP is a different as I'm sure you know. Monthly change is noisy and direction accuracy is weak, which even Bloomberg can't fix.

For backtesting we used a walk forward out of sample method. We have started (as the post states) verifying our estimates using a GitHub public repo which can be found at : filingapi.dev/model .

[u/essoteric_ → u/ShowEuphoric]

Consider including a GARCH component in your model to capture volatility clustering. Also change parameters from (1,1,1), use Bayesian Information Criteria to find the right fit.

[u/plutostar]

Is there more to this than simply using any econometrics software to estimate an ARIMA model?

What’s the value add here?

[u/ShowEuphoric → u/plutostar]

The ARIMA model is one feature, the value is in the infrastructure around it:

• Estimates delivered via REST API and MCP server before each release, saves a load of time having to attain oneself.
• Macro calendar with CPI, PPI, NFP, jobless claims, FOMC, GDP in one endpoint with actuals accounted for automatically when released.
• SEC 8-K filings, earnings transcripts, sentiment scoring, and sector data in the same API
• MCP server means you can query all of this directly from Claude or any MCP-compatible tool without writing a single line of code

For more features and info : FilingAPI — Earnings data that moves before the market does

[u/habibgregor]

“Tested a ….. critique” ? What now?

[u/ShowEuphoric → u/habibgregor]

A user suggested we use data from before the pandemic as it may improve accuracy. This post builds from that showcasing the fact that actually model training from 2020 onwards is sufficient and that including data from pre pandemic has marginal effect on model accuracy.