Download Understanding Confidence Intervals & Hypothesis Testing: Sampling & Test Statistics - Prof and more Study notes Data Analysis & Statistical Methods in PDF only on Docsity!
ISYE 2028 A and B
Lecture 10
Sampling Distributions and Test Statistics
Dr. Kobi Abayomi
April 2, 2009
1 Introduction: The context for Confidence Intervals
and Hypothesis Testing: Sampling Distributions for
Test Statistics
Here is a (non-exhaustive) illustration of the population—sample dichotomy that is the center of what we are studying in this introductory course.
Population Sample Random Variable Statistic Population Mean, Expectation Sample Mean Parameter Estimate μ x
We make assumptions or define a population to ”fit” observed data. Our data is information about events we wish to speak—or gain inference about.
The natural framework is that of an experiment: the population composes our assumptions about what might happen; the sample data compose what we actually observe. Our beliefs about what we see—that is the sample distribution, are related to our general assumptions— that is the population distribution.
We have canonical population models in our overview of random variables. Bernoulli, Bi- nomial, Poisson, Normal, Exponential, etc. characterize types of experiments; we use these characterizations to make statements about data.
Bernoulli distribution to model simple events that can either happen or not. Like whether a coin turns up heads or not.
Binomial distribution to model sums or totals of Bernoulli events. Like whether a coin turns up heads k times in n tosses.
Poisson distribution to model Binomial type events when the probability of any event is very low, and the number of events is very high. Like the number of soldiers who are kicked in the head in a military campaign in 18th century France.
Exponential distribution to model continuous, positive events like waiting times, or time to failure.
Normal distribution to model averages of events, or events where the outcomes are contin- uous, or when we just don’t know any better (ha!).
Chi-Square distribution to model squared deviations, sums of squared deviations, and squared normal random variables.
Moving on, we use our these canonical random variables, to make statements about observed data.
The setup is almost always this: we compare observed data to an expected value under our assumptions. This comparison yields a test statistic.
We then use our probability model (i.e. our fundamental assumption about population for the data) to make a probabilistic statement about the population parameter.
In general, a test statistic looks like this:
T estStatistic =
observed value − expected value standard error
In general the ”observed value” will be some statistic or function of data. The ”expected value” will be some parameter, the population correspondence of the statistic. We call statistics used in this context – to estimate population parameters – estimators. A popular notation is to use θˆ, read ”theta-hat”, as an estimator of the population parameter θ. We have already been exposed to one such estimator: ˆμ = x – the sample mean.
We use functions of data — statistics — to estimate parameters and then our test statistics
are rescalings by the standard deviation of our estimator. We call
V ar(θˆ) = the standard error of the estimator.
2.1.2 Example
We saw that we could use the number of successes in a Binomial experiment as an estimate of the parameter for a Bernoulli.
We let ˆp = Yn and Y ∼ Bin(n, p); remember that Y =
Xi and every Xi ∼ Bern(p).^1
Then ˆp ≡ our estimate for the population proportion of success of the Bernoulli experiment notated with X.
Using some numbers for illustration: It is known that .42 of trick or treating nutritionists are overweight. How likely is it 50 nutritionists, out of a sample of 100 are pleasantly plump?
Solution
First notice that we are given the population proportion p = .42 by the words ”it is known”. Notice also that we have crossed into the world of data - if the word ”sample” is deleted then we could see this as merely a standard Binomial probability question.^2
Here we are asking a question about the distribution of ˆp, the sample estimate of the popu- lation proportion: P(ˆp ≥ .5).
We know that the distribution of the sample estimate of the population propor- tion is normal: ˆp ∼ N (μ =. 42 , σ^2 =.^42100 ∗.^58 ).
We’ll use a z-statistic: zo = x−σ μ= (^) (. 42.^5 ∗−. 58.^42 100 )^1 /^2
Then the probability that we’ll see 50 or more heavy nutritionists out of a sample of 100 is: P(ˆp ≥ .5) = P(Z ≥ zo) = P(Z ≥ 1 .63) ≈ .05.
What do I want you to get from this illustration?
- First: The sampling distribution is truly a distribution - we can answer probability questions about sample data by appealing directly to the sample distribution.
- Second: The distribution of the sample mean is Normal. This is the result of the central limit theorem. Regardless of the distribution of the parameter, if our estimate of it is a sample average (an average of data) we can use the CLT to make probability statements. (^1) this is just the definition of a Binomial experiment (^2) And the answer would be P(Y ≥ 50) = ∑^100 k=50 C k (^100) (.42)k(.58) 100 −k.
- Third: Notice the special use of the Binomial setup to generate estimates of the Bernoulli parameter.
- Fourth: Notice our usual construction of Z so that we can use our standard normal tables (in the back of the book or on your computer).
Situations often arise where the sample mean cannot sufficiently describe, or test for, im- portant hypothetical differences in populations. We must appeal to other distributions, to other quantifications of difference, to test other hypothesis. A useful alternative is...
2.2 The T Distribution
In many situations we cannot assume that we know the variance of the sample mean. As well, we often have not enough samples to apply the central limit theorem to the sampling distribution. In these situation we construct the t-statistic:
t =
x − μ s/
n
The t-distribution, T ∼ t(df ) is an approximation to the normal distribution. Notice I have written df as the parameter of the distribution.^3
The T distribution is centered at zero, just like the Z.^4. We let df ≡ degrees of f reedom.
When we talk about sample data, we loosely define ”degrees of freedom” as the number of independent observations — the number of observations we have left after we subtract the number of parameters we have to estimate.
df ≡ n − k
where we let n = the number of observations and k = the number of parameters to be estimated.
Notice that our constructed t-statistic is a deviation, which we expect to be Normal, rescaled
(or divided) by our estimate of the standard deviation
s^2 n.
Notice that (^3) What, if any, are the parameters for the Z N (0, 1) distribution? The parameters are μ = 0 and σ (^2) = 1. (^4) It turns out that E(T ) = 0 and V ar(T ) = r r− 2
We then need to look at the probability distribution for T.
Example
What is the probability of a sample of size 25 having a mean of 518 grams and standard deviation of 40 grams, if the population mean yield is 500 grams.
Solution
The t statistic is
t =
Then:
P(X > 518) =
= P(t 24 > 2 .25) = 0. 02
3 The Chi Squared Distribution and Test statistic
Example Say we are interested in the fairness of a die. Here is the observed distribution after 120 tosses:
Die Face 1 2 3 4 5 6 Obs. Count 30 17 15 23 24 21
What is the probability that the die is fair?
Using what we only what we have done so far we could test the hypothesis that the die is fair by doing a test of mean:
What is the probability that the mean is 3.5?
We calculate the sample mean to be x = 3.433. Using the variance of a fair die, σ^2 = 2.91, we can compute the sampling distribution and thus the value of the z-statistic is
z 0 =
This yields
P(Z > zo) =. 4364
too high to say it is unfair.
For a better test^5 in this situation I’ll point out that the expected number of counts, for each die face, —under the hypothesis that the die is fair and each face is equally likely— should be 16 ∗ 120 = 20. Looking at our data, it seems we have more than 20 in some cases, and less than 20 in others; the positive and negative deviances tend to cancel out.
Die Face 1 2 3 4 5 6 Obs. Count 30 17 5 23 24 21 Exp. Count 20 20 20 20 20 20
A better test statistic here is the Chi-square^6 statistic, χ^2 :
χ^2 =
∑^ n
i
(obsi − expi)^2 expi
The Chi-squared distribution is strictly positive and takes one parameter, ν - the degrees of freedom, or number of independent observations (number of observations minus the number of parameters to be estimated).
3.1 The Chi-Square test for Goodness of Fit
In general, the Chi-square statistic is a test of Goodness of Fit, or how well the data fits, distributionally. Large values of the Chi-square statistic indicate large deviations of the observed from the expected, thus we reject the null hypothesis for large values of the test statistic. For the Goodness of Fit type hypotheses tests, the deviations have already been squared: the tests are naturally one sided.
The observed count at each bin is obvious and collected in the data. We must calculate the expected count for each bin under the null hypothesis. Here, if the die is fair, the probability of getting in any bin is: P(Die is 1) = · · · = P(Die is 6) = 1/6. So the expected number of counts in each bin is 1/ 6 ∗ 120 = 20. This is the general procedure, if I call P(Bini) = πi, then:
Expectedi = πi ∗ n (^5) More powerful (^6) The χ (^2) distribution, in general, is the sum of squared independent standard normal random variables.
That is χ^2 =
∑n i Zi^ where each^ Z^ ∼^ N^ (0,^ 1). We say^ χ (^2) has n degrees of freedom.
Then P(X = row i and Y = col j) = n ni...nn.j... So the expected number of counts in row i and column j, under a hypothesis of independence, is:
Expectedij =
ni.n.j n..
For our data here we calculate
χ^2 o =
(6 − 11 .89)^2
(75 − 82 .65)^2
And
P(χ^24 > 14 .92) < 0. 005
We conclude that class level and fashion are not independent.
4 F -distribution for ratio of variance
If X 1 , ...Xm is distributed N (μ 1 , σ^21 ) and Y 1 , ...Yn is distributed N (μ 2 , σ^22 ) then the ratio
F =
S 12 /σ^21 S 22 /σ^22
has what we call an F distribution with numerator degrees of freedom m−1 and denominator degrees of freedom n − 1.
F is the ratio of two independent chi-squared variables, call them U ∼ χ^2 (m − 1) and
V ∼ χ^2 (n − 1). If U = (m−1)S
2 σ 12 then^ U^ ∼^ χ
(^2) (m − 1).
If V = (n−1)S
2 σ^22 then^ V^ ∼^ χ
(^2) (n − 1). Then
F =
(m−1)S^2 σ^2 /(m^ −^ 1) (n−1)S^2 σ^2 /(n^ −^ 1)
which just simplifies to (3).^7
An important identity for the F-distribution is:
F 1 −α,ν 1 ,ν 2 = F (^) α,ν−^12 ,ν 1 (4)
You’ll notice that you may have to use this fact in looking up values on the F -table in some books.
5 Miscellanea
5.1 Boxplots
A boxplot is an illustration of the distribution of a a sample
Figure 1 This boxplot displays data that is skewed to the right and with an IQR = 4.
In R:
x<-rnorm(100)
(^7) It turns out that E(F ) = ν 2 ν 2 − 2 and^ var(F^ ) =^
2 ν^22 (ν 1 +ν 2 −2) ν 1 (ν 2 −2)^2 (ν 2 −4) where^ U^ ∼^ χ (^2) (ν 1 ), V ∼ χ (^2) (ν 2 ), F = U/ν 1 V /ν 2 and U is independent of V.
Figure 2 This qqplot displays data from a t distribution with 5 degrees of freedom.
In R
y <- rt(200, df = 5)
qqnorm(y); qqline(y, col = 2)
6 Exercises
- Do exercises 8.39-8.49 on page 265.
- Do exercise 8.57 and 8.61 on page 266.
- Do exercise 8.69 and 8.73 on page 267.
- Use R to make Normal Quantile-Quantile plots for averages of samples drawn from
- Binomial
- Poisson
distributions. Draw samples (say of size 10, 25, whatever) and take averages of these samples. Then make the Q-Q plot on the distribution of averages. Look at boxplots of the distributions of averages.
