Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips

Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Community

Ask the community

Ask the community for help and clear up your study doubts

University Rankings

Discover the best universities in your country according to Docsity users

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Mathematical AnalysisProbability TheoryStatisticsLinear Algebra

A detailed explanation of covariance and correlation between two joint random variables x and y. It includes definitions, identities, proofs, and examples. Covariance is a measure of the linear relationship between two variables and is defined as the expected value of the product of the deviations of the variables from their respective means. Correlation is a normalized version of covariance, which is also a measure of the linear relationship between two variables, but it is bounded between -1 and 1. The document also covers the bilinearity of covariance and the relationship between variance and covariance.

What you will learn

- How is correlation related to covariance?
- What is the identity for covariance and how is it proven?
- What is the definition of covariance between two random variables X and Y?

Typology: Study Guides, Projects, Research

2021/2022

1 / 2

Download Covariance and Correlation: Proof and Identities and more Study Guides, Projects, Research Probability and Statistics in PDF only on Docsity! Covariance and Correlation Math 217 Probability and Statistics Prof. D. Joyce, Fall 2014 Covariance. Let X and Y be joint random vari- ables. Their covariance Cov(X, Y ) is defined by Cov(X, Y ) = E((X − µX)(Y − µY )). Notice that the variance of X is just the covariance of X with itself Var(X) = E((X − µX)2) = Cov(X,X) Analogous to the identity for variance Var(X) = E(X2)− µ2 X there is an identity for covariance Cov(X) = E(XY )− µXµY Here’s the proof: Cov(X, Y ) = E((X − µX)(Y − µY )) = E(XY − µXY −XµY + µXµY ) = E(XY )− µXE(Y )− E(X)µY + µXµY = E(XY )− µXµY Covariance can be positive, zero, or negative. Positive indicates that there’s an overall tendency that when one variable increases, so doe the other, while negative indicates an overall tendency that when one increases the other decreases. If X and Y are independent variables, then their covariance is 0: Cov(X, Y ) = E(XY )− µXµY = E(X)E(Y )− µXµY = 0 The converse, however, is not always true. Cov(X, Y ) can be 0 for variables that are not inde- pendent. For an example where the covariance is 0 but X and Y aren’t independent, let there be three outcomes, (−1, 1), (0,−2), and (1, 1), all with the same probability 1 3 . They’re clearly not indepen- dent since the value of X determines the value of Y . Note that µX = 0 and µY = 0, so Cov(X, Y ) = E((X − µX)(Y − µY )) = E(XY ) = 1 3 (−1) + 1 3 0 + 1 3 1 = 0 We’ve already seen that when X and Y are in- dependent, the variance of their sum is the sum of their variances. There’s a general formula to deal with their sum when they aren’t independent. A covariance term appears in that formula. Var(X + Y ) = Var(X) + Var(Y ) + 2 Cov(X, Y ) Here’s the proof Var(X + Y ) = E((X + Y )2)− E(X + Y )2 = E(X2 + 2XY + Y 2)− (µX + µY )2 = E(X2) + 2E(XY ) + E(Y 2) − µ2 X − 2µXµY − µ2 Y = E(X2)− µ2 X + 2(E(XY )− µXµY ) + E(Y 2)− µ2 Y = Var(X) + 2 Cov(X, Y ) + Var(Y ) Bilinearity of covariance. Covariance is linear in each coordinate. That means two things. First, you can pass constants through either coordinate: Cov(aX, Y ) = aCov(X, Y ) = Cov(X, aY ). Second, it preserves sums in each coordinate: Cov(X1 +X2, Y ) = Cov(X1, Y ) + Cov(X2, Y ) and Cov(X, Y1 + Y2) = Cov(X, Y1) + Cov(X, Y2). 1