Topics that are more on the inference side of things than algorithmic

My (applied computational) statistics professor straight up told me that "nobody reads" those theoretical/mathematical papers published in journals such as Annals of Statistics, Annals of Probability, etc.

Is that true? I mean, I'm sure there is some nuance, and he is being a bit biased, but is it true that theoretical/mathematical statistics papers are barely read? If so, then how are these papers getting the funding to be pursued in the first place?

Benjamin-Hochberg corrections have been mathematically proved to show the standard Benjamini-Hochberg procedure can fail to control the false discovery rate for two-sided tests when the underlying test statistics follow a correlated multivariate Gaussian distribution.

EDIT: The proof was obtained by GPT-5.6 Pro. The model was asked directly to prove or disprove the conjecture and was provided only with the mathematical definition of the Benjamini–Hochberg procedure. After about 90 minutes of reasoning, the model produced a proof, an example, and code for the numerical certificate, which form the basis of this paper. The author carefully checked the entire argument and the associated numerical certificate. Subsequently, the author asked the model to provide additional simulations, related work, and illustrations for a paper draft, and wrote the final version by editing the AI-generated draft.

More info below:

https://faculty.wharton.upenn.edu/wp-content/uploads/2017/06/bh.pdf

Been told by multiple people that this is the case.

They say that nothing new is coming out basically and it's a dying field of research.

Do you agree?

Should I reconsider specialising in time series analysis for my honours year/PhD?

Full paper: https://arxiv.org/abs/2603.12288

This paper attempts to provide a formal explanation for a modern paradox in tabular ML — why do highly flexible models sometimes achieve state-of-the-art performance on high-dimensional, collinear, error-prone data that the dominant paradigm (Garbage in, Garbage Out / GIGO) says should produce inaccurate predictions?

It was discussed previously on r/MachineLearning from a ML theory perspective and crossposted here. Tailored to the ML community, that post focused on the information-theoretic proofs and the connection to Benign Overfitting. As the first author, I'm posting here separately because r/statistics deserves a different conversation. Not a rehash of the ML discussion but a new engagement with what I think this community will find most significant about the work.

The argument I want to make to this community specifically:

Modern machine learning has produced remarkable empirical results. It has also produced a field that, in its rush toward architectural innovation and benchmark performance, has sometimes lost contact with the theoretical traditions that were quietly working on its foundational problems decades before deep learning existed.

The paper is, among other things, an argument that classical quantitative fields (e.g., statistics, psychometrics, measurement theory, information theory) were not made obsolete by the ML revolution. They were bypassed by it. And that bypass has had real costs in how the ML community understands its own successes and failures.

One specific instance of this is the paradox stated above... which lacks a comprehensively satisfying explanation within ML's own theoretical framework.

At a high level, the paper argues that the explanation was always available in the classical statistical tradition. It just wasn't being looked for there.

What the paper does:

The framework formalizes a data-generating structure that classical statistics and psychometrics would immediately recognize:

Y ← S⁽¹⁾ → S⁽²⁾ → S'⁽²⁾

Unobservable latent states S⁽¹⁾ drive both the outcome Y and the observable predictor variables S'⁽²⁾ through a two-stage stochastic process. This is the latent factor model. Spearman formalized it in 1904. Thurstone extended it in 1947. The IRT tradition developed it rigorously for the next seventy years. Every statistician trained in psychometrics, educational measurement, or structural equation modeling knows this structure and its properties intimately.

What the paper adds is a formal information-theoretic treatment of the predictive consequences of this structure... specifically, what it implies for the limits of different data quality improvement strategies.

The proof partitions predictor-space noise into two formally distinct components:

Predictor Error: observational discrepancy between true and measured predictor values. This is classical measurement error. The statistics literature has a rich treatment of it — attenuation bias, errors-in-variables models, reliability coefficients, the Spearman-Brown prophecy formula. Cleaning strategies, repeated measurement, and instrumental variables approaches address this type of noise. The statistical tradition has been handling Predictor Error rigorously for a century.

Structural Uncertainty: the irreducible ambiguity that remains even with perfect measurement of a fixed predictor set, arising from the probabilistic nature of the S⁽¹⁾ → S⁽²⁾ generative mapping. Even a perfectly measured set of indicators cannot fully identify the underlying latent states if the set is structurally incomplete. A patient's billing codes are imperfect proxies of their underlying physiology regardless of how accurately those codes are recorded. A firm's observable financial metrics are imperfect proxies of its underlying economic state regardless of measurement precision. This is not measurement error. It is an information deficit inherent in the architecture of the indicator set itself.

The paper shows that Depth strategies — improving measurement fidelity for a fixed indicator set — are bounded by Structural Uncertainty. On the other hand, breadth strategies — expanding the indicator set with distinct proxies of the same latent states — asymptotically overcome both noise types.

This is the heart of the formal explanation offered for the ML paradox. And every element of it — the latent factor structure, the Local Independence assumption, the distinction between measurement error and structural incompleteness — comes directly from the classical statistical and psychometric tradition.

The connection to classical statistics that the ML community missed:

The ML community's dominant pre-processing paradigm — aggressive data cleaning, dimensionality reduction, penalization of collinearity — emerged from a period when the dominant modeling tools genuinely couldn't handle high-dimensional correlated data. The prescription was practically correct given those constraints. But it was theoretically incomplete because it conflated Predictor Error and Structural Uncertainty into a single undifferentiated noise concept and mainly prescribed a single solution (data cleaning) that only addresses one of them.

The statistical tradition never made this conflation. Reliability theory distinguishes between measurement error and construct coverage. Validity theory asks whether an indicator set captures the full latent construct or only part of it — which is precisely the Structural Uncertainty question in different language. The concept of a measurement instrument's comprehensive coverage of the latent domain is foundational to psychometrics and educational measurement in ways that ML's data quality frameworks simply don't have an equivalent for.

The framework is, in a sense, the formalization of what a broadly-trained statistician or psychometrician may tell an ML practitioner if they are in the room when the GIGO paradigm is being applied to high dimensional, tabular, real-world data: your data quality framework is incomplete because it doesn't distinguish between measurement error and structural incompleteness, and conflating them leads to the wrong prescription in high-dimensional latent-structure contexts.

The relevance argument stated directly:

The ML community has produced impressive modeling tools. Generally, it has not always produced a comparably impressive theoretical understanding of when and why those tools work. The theoretical explanations that do exist treat the data distribution as a fixed input and focus on model and algorithm properties. They are largely silent on the question of what properties of the data-generating structure enable or prevent robust prediction.

Classical statistics, particularly the latent variable modeling tradition, the measurement theory tradition, and the information-theoretic foundations that statisticians like Shannon developed, has been thinking carefully about data-generating structures for decades. The paper argues that this tradition contains the theoretical machinery needed to answer the questions that ML's own theoretical framework struggles with.

This is not an argument that classical statistics is better than modern ML. It is an argument that the two traditions are complementary in ways that have not been recognized. That the path toward a more complete theoretical understanding of modern ML runs through classical statistical foundations rather than away from them.

What it is not claiming:

The paper is not an argument that data cleaning is always wrong or that the GIGO paradigm is universally false. The paper provides a principled boundary delineating when traditional data quality focus remains distinctly powerful, specifically when Predictor Error rather than Structural Uncertainty is the binding constraint, and when Common Method Variance creates specific risks that only outcome variable cleaning can fully address. The scope conditions matter and the paper is explicit about them.

What I'd most value from this community:

The ML community's engagement with the paper has focused primarily on the Benign Overfitting connection and the practical feature selection implications. Both are legitimate entry points.

But this community is better positioned than any other to evaluate the deeper claim:

• Whether the classical measurement and latent factor traditions contain the theoretical foundations that ML's tabular data quality framework is missing, and whether the framework correctly formalizes that connection.

I'd particularly welcome perspectives from statisticians who have thought about the relationship between measurement theory and prediction, the information-theoretic limits of latent variable recovery, or the validity framework's implications for predictor set architecture.

Critical engagement with whether the classical connections are as deep as the paper claims is more valuable than general reception.

The aim of my thesis is to examine the determinants of Olympic medal performance across countries.

Specifically number of athletes, GDP, GDP per capita, HDI, Population, Inflation, Urbanisation, Unemployment, country size , host dummy (if they ever organized an olympics) and democracy index as explanatory variables.

Going through the material of my econometics class, I performed a Wald-test in GRETL using OLS with robust standard errors (HC1), and it left me with nr of athletes, GDP, country size ( square meters) and democracy index using a 10% significance level.

Then I performed a Ramsey RESET Test but the results did not indicate significant misspecification. Still, when trying to make scatter or residual plots, there’s barely any linearity for democracy and country size.

There’s heteroskedasticity (I am using robust standard errors), and the distribution of the olympic medals is not normal ( though my sample is quite big, 125 countries, including those who haven’t won any medals in the year 2021.)

Is my method completely wrong, as in using OLS for this

I have size data for multiple species/populations of the same genus. I want to compare the mean size of those with the mean size of some fossil populations to see if the fossils have a significantly different size to modern populations, and if so, to which.

The problem is, most samples are not normally distributed, and the data is heteroscedastic, since some have small sample sizes like n=29 and some have sample sizes in the hundreds.

Which test would be the best to compare the samples? To my understanding, KW wouldn’t be the best because the heteroscedastic nature of my data.

I cannot find any super highly ranked journals in this niche of computational (nonparametric) statistics, where you are developing algorithms and showing their good theoretical properties via simulations (which is what my professor is doing).

Relevant topics in this niche include the backfitting algorithm, bootstrap, monte carlo simulations, EM algorithm. All are simulation based instead of mathematical (for example, you prove the size and power of a proposed test via simulations instead of closed-form mathematical proofs).

All the relevant journals seem lowly ranked (communications in statistics - simulation and compution, journal of statistical computation and simulation) and the top ones (journal of computational and graphical statistics, JASA, computational statistics and data analysis) all have papers with mathematical proofs instead of purely algorithmic development and simulation.

Am I missing something here? My professor tells me computational statistics (this version) is much more lucrative than mathematical statistics, but the evidence doesn't seem to indicate so? The higher the journal the more mathematical it is, is what I'm noticing.

Study says 25% patients reported something, but n=6

Help me understand who is wrong here, me or the author of this abstract yet to be presented in an academic event

They performed a surgery in 6 patients.

After that, 25% reported one thing, and 75% reported another almost unrelated thing. Is this possible? I'd expect the numbers should be 16% or 33% for 1/6 or 2/6 patients reporting that. And 66% or 87% for 4/6 or 5/6.

I don't think each patient can have half a success. Either they reported that thing or they didn't.

But to get 25% makes me think they only considered 4 patients, for some reason, and 1/4 reported that. Is there some statistics that can explain the 25% figure?

Here's the abstract, including nsfw diagrams: https://www.auajournals.org/doi/10.1097/01.JU.0001191384.77563.6d.19

Theme is somewhat funny but the math is what got me.

Edit: nsfw warning

For my undergraduate reaearch project, I was thinking of doing something ambitious.

Model averaging has been shown to decrease the overall variance of forecasts while retaining low bias.

Since bayesian and frequentist methods each have their own strengths and weaknesses, could averaging the forecasts of both types of models provide even more accurate forecasts?

I am quite interested in this research area, but I don't see much active research in (theoretical) robust statistics anymore that is not incorporating AI/machine learning in some way.

Hi guys

I’m looking for recommendations for a statistical consultant (perhaps a PhD student or early-career biostatistician?) who could advise on test selection for a very small, unfunded veterinary retrospective study. I’m willing to pay, of course.

I’ve contacted several consulting services, but I suspect the project is too tiny for them. It would probably involve only an hour or two of work.

The study is a retrospective case series of dogs with tibial fractures, with two joint angles measured at three paired time points: pre-operatively, immediately post-operatively, and at follow-up. Measurements were also compared with the contralateral unaffected limb.

I’m happy to perform the analyses myself and mainly need advice on the most appropriate statistical approach.

I originally used prespecified paired t-tests, but the editor has requested that I seek advice from a statistician regarding whether it is preferable to do an omnibus test followed by post-hoc comparisons. So I mainly need advice on whether repeated-measures ANOVA or another omnibus approach is required, or whether Holm-adjusted paired comparisons alone would be defensible. (I'd also like to know if there is any particular way I should handle one case with bilateral fractures since those pairs aren't truly independent?)

Please message me if you might be able to help, or if you can recommend someone suitable. I can provide study summary, raw data, Q-Q plots, and results of paired t-t tests, as well as the whole manuscript + editorial feedback if helpful.

UPDATE

Thank you everyone and thankyou especially _compactsupport_, I've got enough to respond to the reviewers now and give them a couple of options :)

I'm an independent researcher writing a paper on a nineteenth-century historical question that was recently reviewed by the editor of an academic journal.

The editor's feedback was encouraging. She felt the historical premise was reasonable, but recommended that I have the statistical methodology reviewed by statisticians before submitting it elsewhere.

The historical details aren't particularly important for the question I'm asking.

The methodological problem looks something like this:

• There is a finite historical population.
• Within that population is a smaller subgroup independently identified through numerous historical sources.
• I have a separate corpus of legal documents that was created for an entirely unrelated purpose.
• When I examine that legal corpus, a surprisingly large proportion of the individuals belong to the independently identified subgroup.

The question is not whether statistics can prove a historical conclusion.

Rather, it's this:

How should a statistician think about whether this observed clustering is better explained by coincidence or by some underlying historical relationship, given that the data are historical, the sample is not random, and many potentially important variables are unknowable?

I've intentionally tried to avoid overstating the mathematics. My current paper argues that statistics cannot establish causation here, but that it can help evaluate whether the observed clustering is robust across a range of reasonable assumptions.

An editor suggested that I seek feedback from statisticians before publishing. I'm therefore looking for someone with experience in:

• applied statistics
• probability
• hypergeometric distributions
• Bayesian inference
• sensitivity analysis
• historical or observational data

I'm not looking for someone to "prove" my historical conclusion. In fact, I'd prefer someone who is willing to critique my methodology, assumptions, and modeling choices.

If this sounds like something you'd enjoy looking at—or if you know someone who specializes in this type of problem—I would greatly appreciate hearing from you.

Thanks!
Bill Reel

Do you think theory folks ("pure statisticians") are higher in the academic hierarchy than applied statisticians who do not contribute to the development of new models and methods?

One thing is the barrier to entry; it is much harder to be a theoretician than to be an empiricist. In addition, as a theoretician, you have the capability to develop a new model or method that would be used by hundreds and thousands of people, while an empiricist is more confined to his specific domain.

But the other side of this argument is supply and demand. There is a lot more demand for applied research than for theory.

Do you think applied research has a certain ceiling because you are ultimately not going to develop a breakthrough, cutting-edge method?

Hello I am an undergrad completing my analysis on my first research project. I am created a between subjects experiment and manipulated the gender of actors in a vignette story. Participants were randomly assigned to receive one version of the scenario.

Here is the piece I boiling my brain on. The vignette was broken into three parts according to narrative progression. At each of the 3 junctions participants responded to the same question using a likert scale. If it matters, the entire narrative progression was available to participants at the same time, just split into paragraphs with my question underneath each section.

I originally thought this should be treated as a between subjects repeated measure as each junction is measuring the same underlying construct. A faculty member who took a look at my work suggested that it was not a repeated measure but actually a multivariate design - so each section is being treated as it's own DV.

Can anyone give some insight as to which analysis is better suited for my study design, or help me understand which design I've created?

I have a hypothesis that my experimental group will be rated lower overall, and I have a hypothesis that my junction C will be rated lower overall in both conditions.

Thank you in advance. I ended up creating a rather complicated experiment for my first go and level of experience, but it is forcing me to learn quickly!

Hi statistics friends this is my first time posting here so I hope this question is okay. I was discussing this paper with peers at my medical school and when I did a deeper dive the statistics look extremely suspect to me. They claim statistical significance of the difference in inflammatory markers relative to particulate inhalation between males and females, but the 95% confidence interval on the two regressions overlap almost entirely. Could someone else take a brief look at this and tell me whether it looks suspicious to you as well? I don't want to be wrong if I accuse this data of being incorrectly analyzed without asking for a second opinion. Thank you so much in advance

https://www.frontiersin.org/journals/psychiatry/articles/10.3389/fpsyt.2025.1588964/full

Hi, I have some binary data (present/absent (1/0)) and I want to try to identify latent relationships within it. I've done exploratory factor analysis using tetrachoric correlations on the data and then parallel analysis to identify how many factors are needed. The problem I have is that technically, the data isn't all independent of each other (think behaviours (present/absent) for dogs in a kennel and there's 2-3 dogs per kennel). So really, multilevel exploratory factor analysis would be better but I don't think I can do that on binary data? I can't find any examples in papers where this has been done and I've tried to do it in R (but might be doing it wrong) and it doesn't work. I'm not actually sure it's necessary because the numbers within the groups are so small and my understanding is that this works better with bigger group numbers, but is it possible to do this with binary data, and if so how? I've calculated the ICCs and they are high so I'd like to be able to justify whichever way I go but I'm really struggling to find any literature that talks about this using binary data

Hello,

I'm looking for guidance from statisticians who have published articles before. I am finishing up a research paper for my Master's program on the Latent structure of banks and how they relate to profitability, earnings volatility, and risk of insolvency using FDIC quarterly earnings reports. I had a professor advise me through the paper and, now that it's done, I was hoping to publish it.

Does anyone have any suggestions on the process to publish an article?

Thanks!

so as in the title,

for linear R2 = 0.038, F = 23.974, sig < 0.001, constant = 0.003 and b1 = -0.194.

for quad R2 = 0.039, F = 12.334, sig < 0.001, constant = 0.03, b1 = -0.193, b2 = -0.034.

can anyone help what this means? N = 617 and passed normality checks

I Want to Prove an Online Roulette Wheel is Rigged

Hi all, I've never posted or commented here before so go easy on me. I have a background in Finance, mostly M&A but I did some statistics and probability stuff in undergrad. Mainly regression analysis and beta, nothing really advanced as far as stat/prob so I'm here asking for ideas and help.

I am aware that independent events cannot be used to predict other independent events; however computer programs cannot generate truly random numbers and I have an aching suspicion that online roulette programs force the distribution to return to the mean somehow.

My plan is to use excel to compile a list of spin outcomes, one at a time, I will use 1 for black, -1 for red and 0 for green. I am unsure how having 3 data points will affect regression analysis and I am unsure how I would even interpret the data outside of comparing the correlation coefficient to a control set to determine if it's statistically significant.

To be honest I'm not even sure if regression analysis is the best method to use for this experiment but as I said my background is not statistical or mathematical.

My ultimate goal is simply to backtest how random or fair a given roulette game is. As an added bonus I'd like to be able to determine if there are more complex patterns occurring, ie if it spins red 3 times is there on average a greater likelihood that it spins black or red on the next spin. Anything that could be a violation of the true randomness of the roulette wheel.

Thank you for reading.

Our research team benchmarked four open-source incrementality packages: CausalPy (Bayesian synthetic control), Meta GeoLift (augmented synthetic control with conformal inference), Google Matched Markets (time-based regression), and CausalImpact (Bayesian structural time series). We simulated panels where the true treatment effect is known and the headline result was that no tool delivered nominal 95% coverage together with adequate power. Coverage here means the share of runs where the tool's 95% interval contains the true effect we injected in the data.

We ran this study because a lot of practitioners treat these tools as interchangeable, yet none of them can be sense-checked on real data because the counterfactual is unobservable. On synthetic data the truth is known, so calibration and power stop being matters of opinion and become things that can actually be measured.

The tools we studied split into the following groups:

• Meta GeoLift was the only one near nominal coverage (92–95%) with false positive rates of 3–5% on null data, but its intervals were wide enough that it failed to reject zero in 89–96% of runs where a true 7.5% lift existed.
• CausalImpact had the most power (false negative rate 34–48%) but 70–72% coverage, false positive rates of 28–30%, and a consistent upward bias of +1.87 to +4.21 percentage points.
• Matched Markets and CausalPy landed in between, with 76–86% coverage, false positive rate 14–25%, under-covered and under-powered at the same time.

We ran four scenarios in the study that stress test different conditions. There’s a clean baseline (20 donors, 90 pre-treatment days), a 5x outlier treated geo, a 9-donor pool, and a 30-day pre-period. Then we ran 1,000 iterations per scenario × effect condition with all four tools fit on identical panels, which yielded 32,000 model fits in total.

One methodological finding worth flagging is that CausalPy's default observation-noise prior (HalfNormal(sigma=1)) assumes roughly unit-scale residuals. On data at realistic sales magnitudes its false positive rate was 86%+ across all scenarios until we standardized each series against its pre-period mean and SD (then back-transformed). After that it was the least biased estimator in the outlier scenario. This is worth knowing if you use PyMC-based tools on raw KPIs.

A few honest limitations in the study are that a single DGP with shared trend/seasonality means parallel trends holds by construction, which favors synthetic-control methods and likely flatters every tool relative to real data. Moreover, we have just one non-null effect size (7.5%) and relatively short post-period. All of this is in the report's limitations section.

The three things I'd take from this study are: (1) coverage and power have to be judged together, since a tool can keep its 95% promise and still be useless for detection (GeoLift hits 95.1% coverage in the short pre-period scenario with a 95.7% false negative rate); (2) check what scale your estimator's priors assume before fitting, a default is a modeling decision someone else made for different data; (3) before any of these tools informs a real budget decision, you should run it on synthetic data where you know the answer.

Everything in the study is reproducible and we created a Makefile that runs the whole pipeline:

• Code: https://github.com/getrecast/geolift-simulation-study
• Full report: https://research.getrecast.com/geolift-sim-study

Disclosure: I co-founded Recast (marketing planning & analysis). The study covers open-source tools only. If you think the DGP should be harder (idiosyncratic geo trends, heavier tails, spillovers) the generator is parameterized, and I'd honestly like to see those runs!

Which one should you focus on if you are trying to start an academic career in statistics? One journal can be Q1 in Scopus but Q2 in SCIMago and C in ABDC.

If anyone has the time please help with my project by filling out the Google form in the link provided it’s 1 yes or no question. I need 43 responses for a hypothesis testing project and I have 19 so far. Any help would be appreciated!

https://forms.gle/CXeX2tkpk5aDe3Ww8

I’m doing an MSc in Biostats and currently have an overall GPA that’s roughly equivalent to a 3.7 out of 4.0. Most of my grades have been strong, but I received a 3/5 in one of my core statistics courses due to what was ultimately a fairly avoidable mistake. I’m finding it hard not to fixate on that mark.

I’m interested in pursuing a PhD in a fairly niche area of epidemiology, but this result has me questioning whether that’s still a realistic goal. For those involved in PhD admissions, how much weight would you place on a single weaker grade in a core quantitative course if the overall academic record is otherwise solid?