Bayesian Estimation and Confidence Intervals
Lecture XXII
I. Bayesian Estimation
A. Implicitly in our previous discussions about estimation, we adopted a classical
viewpoint.
1. We had some process generating random observations.
2. This random process was a function of fixed, but unknown.
3. We then designed procedures to estimate these unknown parameters based
on observed data.
B. Specifically, if we assumed that a random process such as students admitted to the
University of Florida, generated heights. This height process can be characterized
by a normal distribution.
1. We can estimate the parameters of this distribution using maximum
likelihood.
2. The likelihood of a particular sample can be expressed as
22
2
12 21
2
11
, , , exp 2
2
ni
ni
n
L X X X X
3. Our estimates of and
2
are then based on the value of each parameter
that maximizes the likelihood of drawing that sample
C. Turning this process around slightly, Bayesian analysis assumes that we can make
some kind of probability statement about parameters before we start. The sample
is then used to update our prior distribution.
1. First, assume that our prior beliefs about the distribution function can be
expressed as a probability density function where is the
parameter we are interested in estimating.
2. Based on a sample (the likelihood function) we can update our knowledge
of the distribution using Bayes rule
LX
XL X d
D. Departing from the book’s example, assume that we have a prior of a Bernoulli
distribution. Our prior is that
P
in the Bernoulli distribution is distributed
,
.
1. The beta distribution is defined similar to the gamma distribution:
1
1
1
,1
,
f P P P
B
