Download Bayesian Analysis: Obtaining Posterior Marginal Distributions of Parameters - Prof. Mary K and more Study notes Statistics in PDF only on Docsity!
22S:
Bayesian Statistics
Introduction to Multi-Parameter Models
Lecture 9 Sept. 19, 2003
Kate Cowles 374 SH, 335- [email protected]
Multiparameter models
- Real problems in statistics nearly always in- volve more than one unknown quantity.
- However, usually only one, or a few, parame- ters or predictions are of substantive interest.
- Example: newt healing rates
- We may be primarily interested in the pop- ulation mean healing rate Ī¼, but of course we donāt really know the value of the pop- ulation variance Ļ^2.
- So in a realistic model, we must also treat Ļ^2 as an unknown parameter.
- ānuisance parametersā
3
- In cases of this kind, the aim of Bayesian analysis is to obtain the posterior marginal distribution of the parameter(s) of interest, e.g. p(Ī¼|y)
- The general approach is to estimate the joint posterior distribution of all unknown quanti- ties in the model, and then integrate out the one(s) we arenāt interested in.
- Example: in normal means example, we will find p(Ī¼, Ļ^2 |y) then p(Ī¼ | y) =
ā« p(Ī¼, Ļ^2 |y) dĻ^2
4 Example: normal data with both Ī¼ and Ļ^2 unknown
- Need joint prior on both unknown parame- ters.
- Consider first the conventional noninforma- tive prior for this problem
p(Ī¼, Ļ^2 ) ā
Ļ^2
- This arises by considering Ī¼ and Ļ^2 a priori independent and taking the product of the standard noninformative priors for each. - A priori independence may be a reason- able assumption here; it says that if we knew something about one of the unknown parameters, that wouldnāt give us infor- mation about the distribution of the other one.
- Recall standard noninformative priors for Ī¼ when Ļ^2 is assumed known, and for Ļ^2 when Ī¼ is assumed known.
- This is not quite a conjugate prior; we will see that the posterior distribution does not factor like this into an inverse gamma times an independent normal.
- Note that this prior is improper, and the joint posterior is improper if there are fewer than two observations in the current data.
Joint posterior distribution with con- ventional noninformative prior
- The joint posterior is p(Ī¼, Ļ^2 ) ā (^) Ļ^12 Ć (^) (Ļ^12 )n 2 exp(
ļ£« ļ£¬ļ£ā 1 2 Ļ^2
ā^ n i=1(yi^ ā^ Ī¼)
2
ļ£¶ ļ£·ļ£ø
= (^) (Ļ (^21) )n 2 +1 exp
ļ£« ļ£¬ļ£ā 1 2 Ļ^2
ļ£® ļ£Æļ£°ā^ n i=1(yi^ ā^ yĀÆ)
(^2) + n(ĀÆy ā Ī¼) 2
ļ£¹ ļ£ŗļ£»
ļ£¶ ļ£·ļ£ø
= (^) (Ļ (^21) )n 2 +1 exp
ļ£« ļ£¬ļ£ā 1 2 Ļ^2
[ (n ā 1) s^2 + n(ĀÆy ā Ī¼)^2
]ļ£¶ļ£·ļ£ø
where s^2 is the sample variance of the yis:
s^2 =
n ā 1
ā^ n i=
(yi ā yĀÆ)^2
- yĀÆ and s^2 are the sufficient statistics for Ī¼ and Ļ^2.
7
Steps to the marginal posterior distri- bution of Ī¼
- We will use these identities from conditional probability p(Ī¼ | y) =
ā« p(Ī¼, Ļ^2 |y) dĻ^2 =
ā« p(Ī¼ | Ļ^2 , y)p(Ļ^2 |y)dĻ^2
- It can be shown by direct integration (GCSR p. 67-68) that the marginal posterior distri- bution of Ļ^2 is
p(Ļ^2 |y) ā
(Ļ^2 )
n+ 2
exp
ļ£« ļ£¬ļ£¬ļ£¬ ļ£¬ļ£¬ļ£ā(n^ ā^ 1)s
2 2 Ļ^2
ļ£¶ ļ£·ļ£·ļ£· ļ£·ļ£·ļ£ø
- What parametric density is this?
8 The conditional posterior distribution of Ī¼ given Ļ^2
- Use what we already know about the poste- rior mean of Ī¼ with known variance and a uniform prior on Ī¼:
p(Ī¼ | Ļ^2 , y) = N (ĀÆy,
Ļ^2 n
- Again, it can be shown by direct integration (GCSR p. 68-69) that the marginal poste- rior distribution of Ī¼
p(Ī¼ | y) =
ā« p(Ī¼ | Ļ^2 , y)p(Ļ^2 |y)dĻ^2
is a Studentās t distribution with
- mean ĀÆy
- scale parameter s
2 n
- degrees of freedom n ā 1
distributionsā of each unknown given all the other model quantities
- generates a sample path from the Markov chain - at each iteration, generates a realization of each unknown
What does the WinBUGS user have to input?
- model specification in terms of the distribu- tional relationships between observables and parameters - distributions of observables as functions of parameters (likelihood) - prior distributions of parameters
- auxiliary files containing
- data
- initial values for unknowns
WinBUGS output is samples.
- correlated
- of quantities user has requested WinBUGS to āmonitorā - parameters - missing data - functions of either of these
