Lecture Notes on Covariance and Correlation by Will Monroe for CS 109, Study notes of Probability and Statistics

Download Lecture Notes on Covariance and Correlation by Will Monroe for CS 109 and more Study notes Probability and Statistics in PDF only on Docsity! – 1 – Will Monroe CS 109 Lecture Notes #15 July 28, 2017 Covariance and Correlation Based on a chapter by Chris Piech Covariance and Correlation Consider the two plots shown below. In both images I have plotted one thousand samples drawn from an underlying joint distribution. Clearly the two distributions are different. However, the mean and variance are the same in both the x and the y dimension. What is different? Covariance is a quantitative measure of the extent to which the deviation of one variable from its mean matches the deviation of the other from its mean. It is a mathematical relationship that is defined as: Cov(X,Y ) = E[(X − E[X])(Y − E[Y ])] Themeaning of this mathematical definition may not be obvious at a first glance. If X andY are both above their respective means, or if X and Y are both below their respective means, the expression inside the outer expectation will be positive. If one is above its mean and the other is below, the term is negative. If this expression is positive on average, the two random variables will have a positive correlation. We can rewrite the above equation to get an equivalent equation: Cov(X,Y ) = E[XY ] − E[Y ]E[X] Using this equation (and the fact that the expectation of the product of two independent random variables is equal to the product of the expectations) is it easy to see that if two random variables are independent their covariance is 0. The reverse is not true in general: if the covariance of two random variables is 0, they can still be dependent!