Hypothesis Testing: A Framework for Scientific Inquiry, Lecture notes of Statistics

Download Hypothesis Testing: A Framework for Scientific Inquiry and more Lecture notes Statistics in PDF only on Docsity!

Hypothesis Testing

Outline

Hypothesis Testing Formilising the scientific method Example: Famous Court Cases The Western legal system Framework for Hypothesis Testing Defining Terms Tests for Proportion (Proportion and Sign Tests) Example1: IVF Example2: Vegetarianism in Australia An intuitive approach to the Proportion Test Steps for Proportion Test Sign Test

Example: Famous Court Cases

Or this one? (1995, 1997, 2007, 2008) OJ

The Western legal system

In a legal court case, the defendent is either innocent (not guilty) or guilty. But unless the defendent pleads guilty, we never know what the truth is. In fact, even if the defendent pleads guilty, there may be more going on!

Our modern Western legal system is based on the principle of being ‘innocent until proven guilty’ or ‘proof beyond a reasonable doubt’. Hence we assume H 0 : defendent is innocent. Unless there is strong evidence for H 1 : defendent is guilty.

What the pros and cons of this type of legal system?

Other contexts: Context Type I error Type II error H 0 : Patient is healthy Wrong diagnosis Undiagnosed condition H 1 : Patient has cancer Breast Cancer H 0 : iPhone works Wastage for Apple Ruins Apple reputation H 1 : iPhone is faulty

Accusation of a cat

Modelling and evidences

I (^) Modelling: the lamp is not likely to move by itself, but the cat can move around...like...a lot... I (^) Witness: meteorologist confirmed there was no strong wind that day.

Probability and informed decision between hypotheses

I (^) Probability: P(the lamp fell spontaneously|inital hypothesis and witness accounts) = very small. I (^) Decision: we the people find that, based on witness accounts, there is very little evidence to suggest that the lamp fell spontaneously.

Statisticians are prosecutors of scientific discoveries

I (^) The cat example, as ridiculous as it is, showed off an important idea in science: discrimination between competing hypotheses based on evidences. I (^) A statistical test pits two opposing hypotheses against each other and makes an informed decision using probability. I (^) Statistics allow us to infer trends from data to get close to the truth. This process is called statistical inference. I (^) But we might still make mistakes.

RV revisited: RVs are random

I (^) What makes 2 x = 10 same/different to X ∼ Bin(1, 0 .5)? I (^) They are both placeholder for a precise value before some process. I (^) For the former, x = 5 is the only solution! This is an objective truth. I (^) Value for the latter is probabilistic! I (^) Quantifying assumptions for random events is much more difficult than figuring out the solution to 2 x = 10.

Presumption of innocence then prosecute

I (^) The p = 12 assumption is on trial, against the alternative p > 12. I (^) We hypothesise it is true for now. I (^) Assuming the random variable model X ∼ Bin(10, 12 ). I (^) The data xobs = 10 was called as an evidence. I (^) xobs = 10 seems too extreme under all the assumptions. I (^) Using probability, we can calculate P(X ≥ 10) =

( (^1) 2

) 10 . I (^) Under all above assumptions, if the probability seems too “small”. We may want to reject the hypothesis that p = 12.

Statistics doesn’t speak of the truth

BUT! We can never know for sure if p = 12 is true or false!

I (^) There is still a non-zero probability that xobs = 10 was genuinely randomly generated. I (^) We merely made inference of some underlying mechanism based on data. I (^) Philosophical point: there is no truth. Only inferred trends from data.

Defining Terms

H

I (^) The Null hypothesis H 0 is the default hypothesis: what we currently believe to be true. I (^) The Alternate hypothesis H 1 is a new claim about the population. I (^) The hypotheses are commonly articulated in terms of the unknown population parameter. Eg H 0 : μ = 5. I (^) If so, then the alternate hypothesis can take 2 forms: 1 sided (H 1 : μ > 5 or H 1 : μ < 5 ) or 2-sided (H 1 : μ 6 = 5). I (^) How to decide between a 1 or 2 sided test? The decision must not be influenced by the data (‘data snooping’) – we must specify the hypotheses before we do the actual test. Hence, we always use a 2 sided test, unless we have prior evidence (eg a previous report) which suggests a 1 sided test.

A

The assumptions are necessary for the test to be valid. We check whether they appear valid from the sample.

T

I (^) The Test Statistic τ is a random variable, with a distribution which depends on the unknown parameter. I (^) The observed value of the Test Statistic τobs (or τ 0 ) is calculated from the sample. I (^) Look at the distribution of τ to determine what values will argue against H 0 for H 1. I (^) Hypothesis testing involves some theory about the random variable τ where every possible value {τ 0 } counts as some evidence about H 0. The Hypothesis Test weighs up the evidence against H 0 based on the observed value.