Lecture Notes on Data Analysis and Computer | MATH 425, Study notes of Data Analysis & Statistical Methods

Download Lecture Notes on Data Analysis and Computer | MATH 425 and more Study notes Data Analysis & Statistical Methods in PDF only on Docsity!

MATH 425/

Hao Wang

MATH 425/

Outline of the Course

1. Data Aanlysis and Computer

data histogram, data mean, data variance

2. Probability Foundation

Events, sample space, counting rules, conditional prob-

ability, independence, Bayes’ rule, discrete random vari-

ables, binomial distribution, continuous random vari-

ables, normal distribution, central limit theorem

3. Basic Statistics

sampling distributions, type of estimators,point esti-

mation for large sample case, interval estimation for

large sample case

Objective of the Course

This course is the foundation part of the sequence of

MATH 425/525, MATH 426/526, and MATH 427/527.

This sequence can provide your research with useful

Data Analysis

What is statistics?

Statistics is an area of science concerned with the ex- traction of information from numerical data and its use in making inference on the population from which the numerical data are obtained.

What is population?

Definition 1.1 A population or sample space is the set representing all measurements of interest to the inves- tigator

What does it mean making inference?

Example 1.1 Suppose that in a company a quality con- trol person wants to make a decision on whether this shipment of 100,000 batteries is qualified to deliver. The lifetime of a battery is the single quality measure- ment. Therefore,

sample space = { all the lifetimes of this shipment of batteries }

If the lifetime of a battery is longer than 1000 hours, then this battery is qualified. Also if 98% of the bat- teries in a shipment is qualified, then this shipment of batteries is qualified to deliver. Since we can’t check

the batteries one by one, we only can check very limited number of batteries, then we use statistic technique to make inference and decision.

Definition 1.2 a sample is a subset of measurements se- lected from the population of interest.

Example 1.2 Suppose that the U.S. government wants to change a family income related policy which is based on the average of family incomes in U.S.. Therefore,

sample space = { all the family incomes in U.S. }

Eugene’s family incomes is a sample.

Definition 1.3 A population range is an interval from the smallest to the largest measurements in the pop- ulation. If the population range is decomposed into a group of mutually exclusive categories or subintervals, then the frequency is the number of measurements in each category and the relative frequency is the propor- tion of measurements in each category. Definition 1.4 If we equally decompose a population range to get a group of categories or classes, a relative frequency histogram for a data set or population is a bar graph in which the height of the bar represents the relative frequency of a category

Example 1.3 Suppose that in a class there are 10 stu- dents whose final exam scores out of 100 are as follows: 98, 90, 85, 80, 70, 69, 75, 60, 43, 50

their dispersions or variabilities are different. This means that their ways of spreading out are different.

−10^0 −8 −6 −4 −2 0 2 4 6 8 10

Relative

frequency

Normal: mean = 0, variance = 1, size = 300

Relative

 - −10^0 −8 −6 −4 −2 

• 1. - Normal: mean = 0, variance = 4, size = frequency 

(μ − 3 σ, μ + 3σ).

The Tchebysheff’s Theorem applies to any shape of rel- ative frequency histogram (r.f.h.), so it is very conser- vative. The following r.f.h. is not symmetric (skewed).

−1^0 0 1 2 3 4 5 6 7 8

Relative frequency

Example 1. Suppose that the population histogram is symmetric about its mean and the mean and variance of the pop- ulation of n = 108 measurements are 60 and 100, re- spectively. Use Tchebysheff’s theorem to prove that at most 13 measurements are greater than 80. Proof: In Tchebysheff’s theorem, we take k = 2, then at least 3/4 of the measurements lie within [60 − 2 ∗ 10 , 60 + 2 ∗ 10]. In other words, at most 1/4 or 27 of the mea- surements lie outside of [60 − 2 ∗ 10 , 60 + 2 ∗ 10]. Since the sample histogram is symmetric about its mean, at most 13 of the measurements are greater than 60+2∗10 = 80.

If the shape of the r.f.h. is symmetric and bell shaped, (This is just the normal distribution), then we have a better estimation rule. Precisely, if the r.f.h broken line is very close to the curve of function:

f (x) =

σ

2 π

e−

(x−μ)^2 2 σ^2

then we say that the r.f.h. is symmetric and bell shaped. The following figure show you this meaning.

−8^0 −6 −4 −2 0 2 4 6 8

Relative

frequency

For a symmetric and bell shaped r.f.h., we have fol- lowing better estimation rule. Empirical Rule Given a population of measurements that is approxi- mately bell shaped, then we have the following estima- tions: The interval (μ − σ, μ + σ) contains approximately 68% of the measurements.

x ¯ =

n

∑^ n

i=

xi =

∑^25

i=

xi =

s =

n i=1{xi^ −^ x¯}^2 n − 1

i=1 x (^2) i − (

∑ 25 i=1 xi)^2 n n − 1

2 25 24

We can get (¯x − 2 s, x¯ + 2s) = (6. 22 − 6. 994 , 6 .22 + 6.994) = (− 0. 774 , 13 .214). From the original data, 24 measure- ments or (24/25)100 = 96% of the measurements fall in this interval. This is close to the 95% result of Em- pirical rule.