Download Understanding Errors in Hypothesis Testing: Type I and Type II Errors - Prof. David S. Wal and more Study notes Statistics in PDF only on Docsity!
Lesson 12 Errors in Hypothesis Testing
Outline Type I error Type II error Power Examples in the “real world”
Anytime we make a decision about the null it is based on a probability. Recall that we reject the null when it tests a value that is unlikely for the known population. Extreme values are unlikely for the population, but not impossible. So, there is always some chance that our decision is in error.
Note that we will never know whether we know we have made an error or not with our hypothesis test. When running a test, I only know what my decision is about the test, and not the true state of reality. Thus, this discussion on errors is strictly theoretical.
Type I Errors Whenever a value is less than 5% likely for the known population, we reject the null, and say the value comes from some other population.
Notice that we are saying the value is really from another population distribution out there that we don’t know about.
However, some of the time the value really does come from the known population. Notice that even though the value represented is beyond the critical value it still lies under the curve for the normal population. We reject any values in this range, even though they really are part of the known population.
When we reject the null, but the value really does come from the known population a Type I error has been committed. A Type I error, then, happens when we reject the null when we really should have retained it. Note that a Type I error can only occur when we reject the null.
The part of the distribution that remains under the curve for the known population but is beyond our critical value in the region of rejections is alpha (α). When we set alpha we are setting the probability of making a Type I error.
Type II Errors Whenever a value is more than 5% likely for the known population, we retain the null, and say the value comes from the known population. But, some of the time the value really does come from a different unknown population.
Examples of Errors in the “Real World” Another way to think about Type I and Type II errors is to think of them in terms of false positives and false negatives. A Type I error is a false positive, and a Type II error is a false negative.
A false positive is when a test is performed and shows an effect, when in fact there is none. For example, if a doctor told you that you were pregnant, but you were not then it would be a false positive result. The test shows a positive result (what you looking for is there), but the test if false.
A false negative is when a test is performed and shows no effect, when in fact there is an effect. The opposite situation of the above example would apply. A doctor tells you that you are not pregnant, when if fact you are pregnant. The test shows a negative result (what you are looking for is not there), but the test is false.
Let’s look at another example. A sober man fails a blood alcohol test. What type of error has been committed (if any)? For this type of problem you will get two pieces of information. First, whether the test was positive or negative. The test is positive if what you are looking for is found. It is negative if the test shows what you are looking for is not there. The second piece of information is whether the test is in error or not (false or true test). Thus, for this example, the test is positive because if you fail a blood alcohol test it is showing that there is alcohol in your system. You are positive for alcohol in that case. Since the man is sober, the test is false. So, here we have a false positive test or Type I error.
