Homework 8: Bayes Factors and Metropolis Algorithm | Assignments Statistics

564-2007

Homework 8

Assigned 05/23/07

Due 05/30/07

1. Bayes factors: Consider the following two models for y1, . . . , yn:

M0:yi=β0+i

M1:yi=β0+β1xi+i,

where in each case 1, . . . , nare i.i.d. normal(0, σ2). Let the prior distribution for β1be normal(0, τ 2).

(a) Obtain p(y1, . . . , yn|M1, β0, σ2) as follows:

i. Write out the joint distribution of (y1, . . . , yn, β1) given β0and σ2. Collect together terms

that depend on β1.

ii. Write the expression in (i) as some normal distribution for β1multiplied by some stuff that

doesn’t depend on β1. To do this you will have to multiply and divide by something that

may depend on y1, . . . , yn, β0, σ2, τ 2.

iii. Integrate this expression.

(b) Calculate and simplify the following expression.

p(y1, . . . , yn|M1, β0, σ2)

p(y1, . . . , yn|M0, β0, σ2)

Explain why your result makes sense: For what values of y,xis the ratio large? For what what

values of y,xis it small?

to obtain p(M1|y, β0, σ2).

2. Metropolis algorithm: In R, type “library(MASS)” and then “?Pima.tr”. in this problem you will

fit a logistic regression model for type as follows:

Pr(typei= 1|x,β) = exp{β0+β1npregi+β2bpi+β3bmii+β4pedi+β5agei}

1 + exp{β0+β1npregi+β2bpi+β3bmii+β4pedi+β5agei}

I have converted the data to a binary vector yand a design matrix Xin which the predictor variables

have been centered and scaled. These objects are in the Hw8 directory. For this problem, the prior

distribution βwill be such that each component is independent, with β0∼normal(0, sd=10) and β0∼

normal(0, sd=1).

(a) write down a line of code or a function in Rsuch that when you plug in a single value of βit

calculates log p(β) + log p(y|β). Make sure you get this part right.

(b) Implement the Metropolis algorithm using the following proposal mechanism:

beta.star<-rnorm(6,beta,c(.5,rep(.05,5)))

Compute the autocorrelation function, and posterior 2.5,50 and 97.5% posterior quantiles for each

component of β.

(*) Use a multivariate normal proposal distribution that has the same standard deviation you used

in (b), but with a correlation based on your estimate of the posterior correlation. How does the

performance of this proposal distribution compare to that of (b)?

*. Prove that the p(θ|y) is the invariant distribution of the Metropolis-Hastings algorithm. You can mimic

the proof from class for the Metropolis algorithm.

Partial preview of the text

Download Homework 8: Bayes Factors and Metropolis Algorithm and more Assignments Statistics in PDF only on Docsity!

Homework 8 Assigned 05/23/ Due 05/30/

Bayes factors: Consider the following two models for y 1 ,... , yn:

M 0 : yi = β 0 + i M 1 : yi = β 0 + β 1 xi + i,

where in each case 1 ,... , n are i.i.d. normal(0, σ^2 ). Let the prior distribution for β 1 be normal(0, τ 2 ).

(a) Obtain p(y 1 ,... , yn|M 1 , β 0 , σ^2 ) as follows: i. Write out the joint distribution of (y 1 ,... , yn, β 1 ) given β 0 and σ^2. Collect together terms that depend on β 1. ii. Write the expression in (i) as some normal distribution for β 1 multiplied by some stuff that doesn’t depend on β 1. To do this you will have to multiply and divide by something that may depend on y 1 ,... , yn, β 0 , σ^2 , τ 2. iii. Integrate this expression. (b) Calculate and simplify the following expression.

p(y 1 ,... , yn|M 1 , β 0 , σ^2 ) p(y 1 ,... , yn|M 0 , β 0 , σ^2 ) Explain why your result makes sense: For what values of y, x is the ratio large? For what what values of y, x is it small? (c) Suppose that a priori p(M 1 ) = w and p(M 0 ) = 1 − w. Show how you could use the result in (b) to obtain p(M 1 |y, β 0 , σ^2 ).

Metropolis algorithm: In R, type “library(MASS)” and then “?Pima.tr”. in this problem you will fit a logistic regression model for type as follows:

Pr(typei = 1|x, β) = exp{β 0 + β 1 npregi + β 2 bpi + β 3 bmii + β 4 pedi + β 5 agei} 1 + exp{β 0 + β 1 npregi + β 2 bpi + β 3 bmii + β 4 pedi + β 5 agei}

I have converted the data to a binary vector y and a design matrix X in which the predictor variables have been centered and scaled. These objects are in the Hw8 directory. For this problem, the prior distribution β will be such that each component is independent, with β 0 ∼ normal(0, sd=10) and β 0 ∼ normal(0, sd=1).

(a) write down a line of code or a function in R such that when you plug in a single value of β it calculates log p(β) + log p(y|β). Make sure you get this part right. (b) Implement the Metropolis algorithm using the following proposal mechanism: beta.star<-rnorm(6,beta,c(.5,rep(.05,5))) Compute the autocorrelation function, and posterior 2.5,50 and 97.5% posterior quantiles for each component of β. (*) Use a multivariate normal proposal distribution that has the same standard deviation you used in (b), but with a correlation based on your estimate of the posterior correlation. How does the performance of this proposal distribution compare to that of (b)?

*. Prove that the p(θ|y) is the invariant distribution of the Metropolis-Hastings algorithm. You can mimic the proof from class for the Metropolis algorithm.

Homework 8: Bayes Factors and Metropolis Algorithm, Assignments of Statistics

Related documents

Partial preview of the text

Download Homework 8: Bayes Factors and Metropolis Algorithm and more Assignments Statistics in PDF only on Docsity!