Statistics Lectures Notes, Lecture notes of Statistics

Download Statistics Lectures Notes and more Lecture notes Statistics in PDF only on Docsity!

Summary of Chapter 4

Edvards Karlis Kraulis ([email protected])

Visualizations to look at as you read the chapter - link

Discrete vs continuous random variables

Discrete random variables

Countable amount of outcomes, e.g. die rolls

Pr (X = x ) ≥ 0

events are mapped to N numbers

Continuous random variables

Uncountable amount of outcomes within a given range

Pr (X = x ) = 0 - probability of zero at a specific value

Pr (a ≤ X ≤ b) ≥ 0 - we can measure the probability of a

given a range

Probability mass function ( PMF )

For discrete random variables:

F (x ) = Pr (X = x )

Defines the probability of each possible value x of the random

variable.

Sum of PMF = 1!

∞ ∑

k = 0

p k

Bernoulli distribution

Used in case of a binary outcome, e.g. a coin toss.

P(X = x ) = f (x ; p) = p

x ( 1 − p)

1 − x

Where:

p - probability of a success

x - one of the outcomes

x ∈ { 0 , 1 }

Binomial distribution

Used for multiple binary outcomes with the same properties,

e.g. multiple coin tosses

P(S n = k) = f (k; n , p) = (

n

k

)p

k ( 1 − p)

n − k

Where

n

k

n!

k !( n − k )!

n - number of trials

k - number of successes

p - probability of a success

Expected value

The mean outcome that a given distribution tends towards.

μ = E(X ) =

∞ ∑

k = 0

kp k

∞ ∑

k = 0

k Pr(X = k)

Some properties

1 E(c) = c

2 E(cX ) = cE(X )

3 E(X + Y ) = E(X ) + E(Y )

Where

X , Y - random variables

c - a given constant

Variance

A measure of spread in realizations. Measured as squared distances

from the mean.

σ

2 = Var(X ) = E(X − μ )

2

∞ ∑

k = 0

(k − μ )

2 p k

Some properties

1 Var(X ) ≥ 0

2 Var(X + a) = Var(X )

3 Var(aX ) = a

2 Var(X )

4 Var(X ) = E

(

X

2

)

− [E(X )]

2

• can be useful to speed up

computations

Where

X - random variables

a - a given constant

Overview of some of the common moments

Distributions of continuous random variables

Probability density function (PDF)

Pr(a ≤ X ≤ b) =

∫ b

a

f (x )dx

Probabilities are determined by area under the curve!

Cumulative density function (CDF)

F (x ) =

∫ x

−∞

f (t)dt = Pr(X ≤ x )

Some properties

(^1) if x 1 < x 2 , F (x 1 ) ≤ F (x 2

2 lim x →−∞ F (x ) = 0

(^3) lim x →∞ F (x ) = 1

Normal distribution

Bell shaped distribution. Arguably the most important distribution

to know because of the central limit theorem (discussed next

chapters).

f (x ; μ, σ ) =

σ

2 π

exp

(

(x − μ )

2

2 σ

2

)

Standardization - remember as this comes up a lot!

Z = (X − μ ) /σ

Standard normal density function (obtained after standardization)

In such case case: μ = 0, σ

2 = 1!

φ (x ; ) = exp

(

−x

2 / 2

)

2 π

Properties of continuous random variables

E ψ (X ) =

∫

R

ψ (x )f (x )dx

Further Moments

ψ (x ) = (x − μ )

p

• further central moments

ψ (x ) = (x )

p

Their existence may depend on the particular distribution

Data Statistics

Chapter 5: Estimation

Summary of the Lecture

Definition of sample statistics;

Distribution of sample statistics;

The central limit theorem;

Confidence intervals;

Estimation procedure: maximum likelihood.

2