Matrix Notation in Statistical Models: OLS Estimator, Variance-Covariance, R-square, Lecture notes of Statistics

Download Matrix Notation in Statistical Models: OLS Estimator, Variance-Covariance, R-square and more Lecture notes Statistics in PDF only on Docsity! Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Precept 5: Simple OLS Soc 500: Applied Social Statistics Simone Zhang1 Princeton University October 2016 1This draws material from Matt Blackwell. Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Today’s Agenda Basic matrix operations Review matrix notation for linear regression Notation OLS estimation Variance-covariance matrix R-square F-test Bootstrap Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Variance-Covariance Matrix The homoskedasticity assumption is different: var(u|X) = σ2 uIn In order to investigate this, we need to know what the variance of a vector is. The variance of a vector is actually a matrix: var[u] = Σu =  var(u1) cov(u1, u2) . . . cov(u1, un) cov(u2, u1) var(u2) . . . cov(u2, un) ... . . . cov(un, u1) cov(un, u2) . . . var(un)  This matrix is symmetric since cov(ui , uj) = cov(uj , ui ) Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Matrix Version of Homoskedasticity Once again: var(u|X) = σ2 uIn In is the n × n identity matrix Visually: var[u] = σ2 uIn =  σ2 u 0 0 . . . 0 0 σ2 u 0 . . . 0 ... 0 0 0 . . . σ2 u  In less matrix notation: var(ui ) = σ2 u for all i (constant variance) cov(ui , uj) = 0 for all i 6= j (implied by iid) Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Sampling Variance for OLS Estimates Under assumptions 1-5, the sampling variance of the OLS estimator can be written in matrix form as the following: var[β̂] = σ2 u(X′X)−1 This matrix looks like this: β̂0 β̂1 β̂2 · · · β̂K β̂0 var[β̂0] cov[β̂0, β̂1] cov[β̂0, β̂2] · · · cov[β̂0, β̂K ] β̂1 cov[β̂0, β̂1] var[β̂1] cov[β̂1, β̂2] · · · cov[β̂1, β̂K ] β̂2 cov[β̂0, β̂2] cov[β̂1, β̂2] var[β̂2] · · · cov[β̂2, β̂K ] ... ... ... ... . . . ... β̂K cov[β̂0, β̂K ] cov[β̂K , β̂1] cov[β̂K , β̂2] · · · var[β̂K ] Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Sum of Squares 1 2 3 4 5 6 7 8 6 7 8 9 10 11 12 Total Prediction Errors Log Settler Mortality Lo g GD P pe r c ap ita g ro wt h Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Sum of Squares 1 2 3 4 5 6 7 8 6 7 8 9 10 11 12 Residuals Log Settler Mortality Lo g GD P pe r c ap ita g ro wt h Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping R-square Coefficient of determination or R2: R2 = SStot − SSres SStot = 1− SSres SStot This is the fraction of the total prediction error eliminated by providing information in X. Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Bootstrap: Intuition Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Simple Example with Sample Means Let Xi = {3, 7, 9, 11, 150} Bootstrapped Samples: X̄boot Xboot,1 3 3 9 11 3 5.8 Xboot,1 7 150 11 7 11 37.2 Xboot,1 11 9 9 7 3 7.8 ... Matrix Notation OLS Estimator Variance-Covariance R-square F Test Bootstrapping Bootstrapped Standard Error Bootstrapped Standard Error sd(X̄boot) Bootstrapped Confidence Interval: Take the 2.5% and 97.5% quantiles of X̄boot