Download Lecture 8 on Statistical Theory I: Moments and Moment Generating Functions and more Study notes Biostatistics in PDF only on Docsity!
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
1
Review for the previous lectureDefinitions:
mean or expected value of a random variable
Examples:
How to calculate the pdf of function and mean of a random variable
Chapter 2 – Transformations and Expectations
Section 2.0 – Basics of Supremum and InfimumDefinition:
For any set
(^
,^
A
, the
supremum
of
A
, denoted by sup
A
, is defined as: (1)
x
A
,^
sup
x
A
ε ∀
,^
0 x
A
, such that
0
sup
A
x
Definition:
For any set
(^
,^
A
, the
infimum
of
A
, denoted by inf
A
, is defined as: (1)
x
A
,^
inf
x
A
,^
0 x
A
, such that
0
sup
x
A
Theorem:
For any set
(^
,^
A
, if
M, such that x
M
for x
A
, then
sup
A
exists.
Theorem:
For any set
(^
,^
A
, if
M, such that x
M
for x
A
, then
inf
A
exists.
Definition:
If sup
A
inf
A
) does not exists, then sup
A
inf
A
Theorem:
If X is a random variable with the continuous cdf
X F
x. Define
)^
inf{
:^
X^
X
F
y
x
F
x
y
−^
for
y <
, then
1
(^
(^
X^
X
F
F
y
y
−^
, then you can use this to prove
X F
y −^
is an increasing function of
y
y <
Section 2.2 – Expected Values
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
2
Theorem 2.2.5:
Let X be a random variable and let a , b , and c be constants. Then for any functions
g
x
and
2
g
x
whose expectations exist, we have
a.
1
2
1
2
(^
(^
)^
(^
)^
)^
(^
)^
(^
E
ag
X
bg
X
c
aEg
X
bEg
X
c
b. If
g
x
for all x , then
)^
Eg
X
c. If
1
2
g
x
g
x
for all x , then
1
2
(^
)^
(^
E
g
X
Eg
X
d. If
a
g
x
b
for all x , then.
a
Eg
X
b
Proof: (discrete case) a.
1
2
1
2
1
2
1
2
(^
(^
)^
(^
)^
)^
(^
)^
(^
(^
)^
(^
)^
(^
(^
)^
(^
)^
x x^
x^
x
E ag
X
bg
X
c
ag
x
bg
x
c P X
x
ag
x P X
x
bg
x P X
x
cP X
x
aEg
X
bEg
X
c
∈ ∈
∈
∈
X X
X
X
b.
1
1
(^
)^
(^
)^
x
Eg
X
g
x P X
x
∈
X
c. If
1
2
g
x
g
x
for all
x
, then
1
2
g
x
g
x
for all
x
, thus
1
2
1
2
(^
E
g
x
g
x
Eg
x
Eg
x
d. Similar as the proof of c. Example 2.2.6 (Minimizing distance):
Find the value
b
that minimizes
2
(^
E
X
b −
Solution 1:
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
4
Definition 2.3.1:
For each integer
n
, the
n th moment
of
X
(or
(^
X F
X
' n
is
'^
n
n^
EX
. The
n th central
moment
of
X
n
is
(^
n)
n^
E X
, where
' 1
E
X
Note:
The first moment of
X
is the mean, i.e.,
' 1
EX
Definition 2.3.2:
The
variance
of a random variable
X
is its second central moment,
2
2
(^
VarX
E X
EX
The positive square root of
VarX
is the
standard deviation
of
X
Question:
What is the first central moment of
X
1
(^
)^
E X
EX
Some Notes:
variance and standard deviation are measures of spread;
always greater than or equal to 0;
When does the variance=0?
An alternate formula for the variance is
2
2
(^
VarX
EX
EX
Example 2.3.3: (Exponential variance)
Let
X
have an exponential distribution. We found that the
E
X
μ
λ
Find the
VarX
Solution:
2
2
/^
2
/
0
0
2
/^
/^
/^
2
0
0
0
|^
x^
x
x^
x^
x
d
EX
x
e
dx
x
e dx
x e
xe
dx
x
e
dx
λ
λ
λ
λ
λ
∞^
∞
−^
−
∞^
∞
−^
∞^
−^
−
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
5
Therefore,
2
2
2
2
2
(^
)^
VarX
EX
EX
Theorem 2.3.4:
If X is a random variable with finite variance, then for any constants a and b , then 2
(^
Var aX
b
a VarX
Proof:
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
(^
)^
(^
)^
(^
(^
(^
)^
(^
(^
)^
(^
)^
Var aX
b
E aX
b
E aX
b
E a X
abX
b
aEX
b
a EX
abEX
b
a
EX
abEX
b
a EX
a
EX
a VarX
Example 2.3.5: (Binomial variance)
Let
X
~Binomial(n, p). We found that
E
X
np
. Find
VarX
Solution:
2
2
2
0
1
1
1
1
1
0
1
1
(^
(^
0
0
(^
)^
)^
(^
)^
(^
(^
)^
)^
n^
n^
n
x^
n^
x^
x^
n^
x
x^
x^
x
n^
y^
n^
y
y
n^
n
y^
n^
y^
y^
n^
y
y^
y
n
n
EX
x P X
x
x
p
p
nx
p
p
x
x
n
n
y
p
p
y
x
y
n
n
np
y
p
p
p
p
y
y
np
n
p
−^
−
=^
=^
=
−^
+^
− −
=
−^
−
−^
−^
−^
−
=^
=
^
^
^
^
^
^
^
^
^
^
^
^
^
^
^
^
^
^
Therefore,
2
2
2
2
2
(^
)^
(^
VarX
EX
EX
n n
p
np
n p
np
p
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
7
Solution:
/(^
)
1
/^
1
1
0
0
(^
)^
(^
(^
(^
x
tx
tx
x^
t
Ee
e x
e
dx
x
e
dx
t^
t
β
α
β
α
β
α
α
α^
α
α
−
∞
∞
−^
−^
−^
−
−
Example 2.3.9 (Binomial mgf):
Find the mgf of a Binomial ( ,
n p
random variable
X
Solution:
0
0
)^
(^
(^
n^
n
tx
x^
n^
x^
t^
x^
n^
x
X^
x^
x
t^
n n
n
M
t^
e
p
p
pe
p
x
x
pe
p
−^
−
=^
=
^
^
^
Note:
Characterizing the set of moments is not enough to determine a distribution uniquely because there may be
two distinct random variables having the same moments but different distributions. (See Example 2.3.10) Theorem 2.3.11:
Let
X F
x
and
(^
Y F
y be two cdfs all of whose moments exist.
a.
If X and Y have bounded support, then
X^
Y
F
u
F
u
for all u is and only if
r^
r
EX
EY
for all integers
r^
b.
If the mgfs exist and
X^
Y
M
t^
M
t
for all for
t
in some neighborhood of 0, then then
X^
Y
F
u
F
u
for all u.
Lecture 8 on BST 631: Statistical Theory I – Kui Zhang, 09/14/
8
Theorem 2.3.12:
(Convergence of mgfs) Suppose
,^
i X
i^
is a sequence of random variables, each with mgf
( )i X M
t. Furthermore, suppose that
lim
i
i^
X^
X
M
t^
M
t
→∞
for all t in a neighborhood of 0, and
X M
t^
is an mgf.
Then there is unique cdf
X F whose moments are determined by
X M
t and for all x where is continuous, we have
lim
i
i^
X^
X
F
x
F
x
→∞
. That is, convergence, for
t^
h <
of mgfs to an mgf implies convergence of cdfs.
Example 2.3.13: (Poisson approximation to Binomial)
Show that if
X
Binomail n p
and
(^
Y
Poisson
np
, then that the cdf of
X
converges to the cdf of
Y
Solution:
[^
)]
t^
n
X M
t^
pe
p
0
0
/^
!^
(^
!^
t
tx^
x^
t^
x^
e
Y^
i^
i
M
t^
e e
x
e
e
x
e
e
λ
λ
λ^
λ
∞
∞
−^
−^
−
=^
=
and we have
(^
lim
[^
)]
(^
t
n
t^
n^
t^
n^
e
X M
t^
pe
p
e
e
n
λ
−
→∞
Theorem 2.3.15:
For any constants
a
and
b
, the mgf of the random variable
aX
b
is given by
(^
bt
aX
b^
X
M
t^
e M
at
+^
Proof:
(^
)^
(^
)^
(^
)
(^
)^
(^
)^
(^
)^
(^
aX
b t
bt
at
X
bt
at
X
bt
aX
b^
X
M
t^
E e
E e e
e E e
e M
at
+^
