Chapter 3. Discrete Random Variables, Slides of Discrete Mathematics

Download Chapter 3. Discrete Random Variables and more Slides Discrete Mathematics in PDF only on Docsity!

Chapter 3. Discrete Random Variables

Review

Discrete random variable:

A random variable that can only take finitely

many or countably many possible values.

Distribution: Let

x 1 , x

2 ,...

be the possible values of

X

. Let

P

X

x i ) =

p i ,

where

p i ≥

0 and

i

p i = 1

Graphic description: (1) probability function

p i } , (2) cdf

F

Tabular form:

x i x 1 x 2

p ( x

i )

p 1

p 2

· · ·

Theorem

: Consider a function

h

and random variable

h ( X

). Then

E

[

h ( X

)] =

∑ i h ( x i

P

X

x i )

Corollary: Let

a, b

be real numbers. Then

E

[

aX

(^) b ] =

aEX

b .

Corollary: Consider functions

h 1 ,... , h

k

. Then

E

[

h

1 ( X

) +

(^) h

k ( X

)] =

E

[

h 1 ( X

)] +

E

[

h k ( X

)] .

Variance (“

σ 2 ”) and Standard deviation (“

σ ”):

Var[

X

]

E

[

X

EX

2 ] ,^

Std[

X

]

Var

X.

Example (interpretation of variance): A random variable

X

P

X

P

X

a ) = 1

P

X

a ) = 1

Examples

1. Suppose one word is randomly selected from the sentence THE GIRL PUT

ON HER BEAUTIFUL RED HAT.

X

denote the number of letters in the

word that is selected.

1. Consider a random variable

X

with mean

μ

and standard deviation

σ

. Its

standardization is

Y

X

μ

σ

What is the mean, variance, and standard deviation of

Y

1. (Portfolio Optimization) Consider a stock whose price today is $10. At the

end of year, its price

S

has the following distribution.

10

6 10 20

1 / 3 1 / 3

1 / 3

2

0 0 6

1 / 3 1 / 3

1 / 3

Stock price

Option

S

( S − (^) 14)

A stock call option is priced at $2 today, with payoff (

S

. You have

portfolio at the end of the year while the expected return is at least 20%.in total $10K for investment. The goal is to minimize the variance of your

Example

1. Toss a fair

n

times. Number of heads? Number of tails?

1. A jumbo jet has 4 engines that operate independently. Each engine has

probability 0.002 of failure.

At least 2 operating engines needed for a

successful flight. Probability of an unsuccessful flight? (Approx. 3

×

− 7 )

1. Toss a coin

n

times, and get

k

heads. Given this, what is the probability

that the first toss is heads?

Some comments on random sampling

1. In a family of size 10, 60% support Republican and 40% support Democrat.

the Republican. Is its distributionRandomly sample 3 members of the family. The number of those support

B

1. In a population of size 300 million, 60% support Republican and 40% support

those support the Republican. Is its distributionDemocrat. Randomly sample 3 members of the population. The number of

B

Remark

: Unless specified, the size of population is always much larger com-

cally distributed. (This comment applies to general random sampling).pared to the sample size. Samples can be regarded as independent and identi-

Estimating

p : very preliminary discussion

Illustrative example.

Pick a random sample of

n

= 100 Americans, and

X

is the number of people support Republican.

What is your estimate for the

Comment. Denote the quantity we wish to estimate bypercentage of the population that support Republican?

p

. It is a fixed num-

ber.

X

has distribution

B

(^) p ).

A natural estimate is to use the sample

percentage

ˆp

.

X

ˆp is a random variable. (If

X

happens to be 50, ˆ

p takes value 0

1. If

X

happens

to be 52, ˆ

p

takes value 0

52, etc).

E

(^) ˆp

=

p,

Var[ˆ

p ] =

p (

p )

n

ˆp

is unbiased and consistent (more on these later).

Maximum likelihood estimate (MLE)

Consider the same example with sample size

n , and

X

is the number of people

support Republican. Then

X

is

B

( n ; (^) p ).

MLE: Suppose

X

k

. What value of

p

makes the actual observation (

X

k )

Solution:most likely?

Maximizing (with respect to

p )

P

X

k ) =

k n

)

p k (

(^) −

p ) n − k.

amounts to maximizing

k ln (^) p (^) + (

n

−

(^) k ) ln(

p ). Check that it is maximized

at

p

=

k/n

. The MLE is

ˆp

n X

.

Remark. In general, MLE is not unbiased but consistent. More on MLE later.

Expectation and Variance of Geometric Distribution

Theorem. Suppose a random variable

X

is geometrically distributed with prob-

ability of success

p

. Then

E

[

X

] =

p 1 ,

Var[

X

] =

p

p 2

Proof. Direct computation.

Examples

1. Memoryless property: Suppose

X

is a geometric random variable with prob-

ability of success

p

. Then for any

n

and

k ,

P

X > n

(^) k

| X > n

P

X > k

1. Toss a pair of 6-side fair dice. What is the probability that a sum of face

value 7 appears before a sum of face value 4?

Poisson distribution

Poisson random variable. A random variable

X

takes values in

such that

P

X

k ) =

e − λ (^) λ k

k ! (^).

We say

X

has Poisson distribution with parameter

λ .

1. What is the physical meaning of this distribution?1. Does this really define a probability distribution? Answer: Yes.

Answer:

Limit of

Binomial distributions

B

n ; (^) p ) with

np

λ , as

n

→ ∞

Remark: Poisson approximation of Binomial

B

n ; (^) p ) when

n

big,

p

small,

and

λ

=

np <

Consider a unit time interval, andAnother story

X

the number of certain events that occur

1. Split the unit interval intoduring this interval.

n

intervals of equal length.

1. On each interval of length2. The occurrence of the event is independent from interval to interval.

s

(small),

P

(^) (1 event occurs)

λs

P

(^) (more than 1 event occurs)

P

(^) (0 event occurs)

(^) λs.

1. Total number of events is approximately

B

n ; (^) λ/n

1. Let

n

go to infinity.