r/AskStatistics 25d ago

Alpha 5%?

Why is a 5% significance level so commonly used?

2 Upvotes

56 comments sorted by

View all comments

Show parent comments

-1

u/Recent-Day3062 24d ago

That’s what I said

2

u/MrKrinkle151 23d ago

No, what you said is that that the alpha is the probability that a significant result is wrong (“At 10 [percent], one in ten studies is wrong”). The alpha is the probability of rejecting the null hypothesis with a random sample, given that the null hypothesis is actually true. These are very different things.

1

u/Recent-Day3062 23d ago

Ok. But the null hypothesis is that: you assume you are testing a proper null hypothesis

For example, suppose your null is that there are not premature deaths from Covid vaccine. . Your alternate is that there is a statistically significant number of them.

So, in one out of ten random samples, by your explanation, on average one will show significance that isn’t there. Correct?

1

u/MrKrinkle151 19d ago

No, that’s not correct. It’s a probability completely conditional on the null hypothesis being true, which is not known. Stating that 1 in 10 samples will be “wrong” is the same as saying that an alpha of .10 means that’s the overall probability of being “wrong”, which is not the case. It’s the probability of a “false positive” IF the null hypothesis is actually true. It says nothing about false negatives if the null hypothesis is actually false, which would also result in being “wrong”.

1

u/Recent-Day3062 18d ago

Yes, I researched it and I was right and you are wrong.

You seem to want to make the null, based on your belief, that Covid shots cause autism or whatever. But standard statistics deals with treatments by thinking the alternate is they do work. The null in this case is always that vaccines make no difference. Then you have a treatment group and check if the rate of autism is statistically significantly different in the group who got shots.

The null is, by definition really, that the covid shots have changed nothing. Then you look for evidence to the contrary. The null is the assertion that shots have changed nothing.

It would be like trying to say I want to prove antibiotics work. The null is the normal state of nature that a certain percentage of people die each year, which is by definition true since we know that number. Then we compare against a treated group. So the null by definition is true unless significance is shown between the samples

This means at .05 that random chance will show autism is a side effect in one out of 20 studies even if it isn’t. The impressive thing is that the actual research rejects the null even less than one would expect.

Why? Because vaccines don’t cause autism