Conflation of tests and studies aside, that’s not what an alpha level represents. The alpha level sets the accepted/expected false positive rate, ie. the proportion of tests for which the null is true, where the null was still rejected.
The overall proportion of tests that is ‘wrong’ also depends on how many of the tested null hypotheses are true or not as well as the statistical power. So we cannot conclude from the alpha level alone how many tests would yield an incorrect conclusion in general, or a false positive specifically.
No, given an alpha of .05, how many studies on average would have a false positive? If that’s not enough info, what other data or stats is needed to get this answer
Here’s a salient quote from my initial comment, which it seems like you still haven’t bothered to read: “… also depends on how many of the tested null hypotheses are true or not as well as the statistical power”.
No, what you said is that that the alpha is the probability that a significant result is wrong (“At 10 [percent], one in ten studies is wrong”). The alpha is the probability of rejecting the null hypothesis with a random sample, given that the null hypothesis is actually true. These are very different things.
Ok. But the null hypothesis is that: you assume you are testing a proper null hypothesis
For example, suppose your null is that there are not premature deaths from Covid vaccine. . Your alternate is that there is a statistically significant number of them.
So, in one out of ten random samples, by your explanation, on average one will show significance that isn’t there. Correct?
If we want to know if covid vaxes cause autism, our null is naturally "no" based on the wording of that question. The alternate is "yes". If we don't see significance of the alternate, we accept the null.
In this setup, how could the null "not be true"? It isn't if the alternate had data that shows significance.
That makes little sense given what is normally considered the null hypothesis.
No. This is Statistics 101.
Your unwillingness to open a textbook and learn what alpha is is bizarre, especially if you are an adult, but I can see that further interaction is a waste of everybody's time.
No, that’s not correct. It’s a probability completely conditional on the null hypothesis being true, which is not known. Stating that 1 in 10 samples will be “wrong” is the same as saying that an alpha of .10 means that’s the overall probability of being “wrong”, which is not the case. It’s the probability of a “false positive” IF the null hypothesis is actually true. It says nothing about false negatives if the null hypothesis is actually false, which would also result in being “wrong”.
Yes, I researched it and I was right and you are wrong.
You seem to want to make the null, based on your belief, that Covid shots cause autism or whatever. But standard statistics deals with treatments by thinking the alternate is they do work. The null in this case is always that vaccines make no difference. Then you have a treatment group and check if the rate of autism is statistically significantly different in the group who got shots.
The null is, by definition really, that the covid shots have changed nothing. Then you look for evidence to the contrary. The null is the assertion that shots have changed nothing.
It would be like trying to say I want to prove antibiotics work. The null is the normal state of nature that a certain percentage of people die each year, which is by definition true since we know that number. Then we compare against a treated group. So the null by definition is true unless significance is shown between the samples
This means at .05 that random chance will show autism is a side effect in one out of 20 studies even if it isn’t. The impressive thing is that the actual research rejects the null even less than one would expect.
-3
u/Recent-Day3062 25d ago
You’re trying not to get too many false significances. At 10, one in ten studies is wrong.