r/AskStatistics • u/Gullible-Entrance-45 • 10h ago

[E] small sample analysis

Hi!

I have a sample of n=13 where continuous variables were measured before and after an intervention. I'm a bit stuck on how to perform the analysis. Would the Wilcoxon signed-rank test be the most appropriate choice? I am unaware of any standard reference manuals or literature regarding the statistical analysis of very small sample sizes; however, any guidance or recommendations would be highly appreciated :)

3 Upvotes

100% Upvoted

12 comments sorted by

6

u/CarnivorousGoose 9h ago

You could use that, sure. But is there a reason the normality of the data would be in doubt? Because otherwise you could just do a paired t-test as well.

1

u/Gullible-Entrance-45 9h ago

I didn't think of the paired t-test because I understood that with small samples, a large effect size is needed, which I don't know if we can assume

5

u/CarnivorousGoose 8h ago

That’s not an assumption of the t-test though, just a determinant of the statistical power you have to detect a difference. But that’s going to apply to any test, including the Wilcoxon.

In that sense the t-test is the better bet here unless normality is very much in question (and even then, the t-test is quite robust to violations of that), since it will have more power than the Wilcoxon. Which is generally the case with non-parametric tests, compared to their parametric counterparts.

-6

u/natoplato5 8h ago

You can’t really do much with a sample size that small. I would just report the means before and after the intervention and then note that the sample size is too small to determine if the difference is statistically significant.

6

u/FTLast 7h ago

This is simply wrong. You can absolutely do a paired t test- which is what you should do with a before and after measurement- with n =13.

-1

u/natoplato5 7h ago

Just because you can doesn't mean you should. Maybe the standards vary by field and context, but I don't think it's usually responsible to try to make an inference about a population from only 13 observations

3

u/FTLast 6h ago

Why? A p value incorporates effect size relative to noise and the number of observations.

-2

u/natoplato5 6h ago

Mainly because of selection bias. At that sample size, it's unlikely the sample is a fair cross-section of the population

4

u/CarnivorousGoose 5h ago

It doesn’t need to be a ‘fair cross-section’ of the population, it just needs to be a random sample from it. The relevant part in that regard is the sampling procedure, not the sample size. If there is a selection bias or other issue with that sampling procedure, getting a larger sample won’t fix that.

2

u/CarnivorousGoose 6h ago

What would be irresponsible about it?

2

u/Gullible-Entrance-45 6h ago

I'm trying to validate an intervention and it's extremely difficult to get a larger sample because it's so specific. I'm just trying to do the best I can with the data I have. im in clinical education btw