Covariance and Correlation - Computer System Modeling Fundamentals - Lecture Slides, Slides of Java Programming

Download Covariance and Correlation - Computer System Modeling Fundamentals - Lecture Slides and more Slides Java Programming in PDF only on Docsity! Today A bit more about joint PMFs •  Variance of sums of independent random variables •  Covariance and correlation Continuous random variables •  Probability density functions (PDFs) •  Uniform continuous random variables •  Exponential random variables •  Cumulative distribution functions (CDFs) Docsity.com Independence •  X and Y are independent if and only if for all k and l pX,Y(k,l) = pX(k) pY(l) Docsity.com Why is independence useful? If X1, X2, …, Xn are all independent, then E[X1X2…Xn] = E[X1]E[X2]…E[Xn] Docsity.com Why is independence useful? If X1, X2, …, Xn are all independent, then E[X1X2…Xn] = E[X1]E[X2]…E[Xn] and var(X1 + X2 + … + Xn) = ??? Docsity.com Why is independence useful? If X1, X2, …, Xn are all independent, then E[X1X2…Xn] = E[X1]E[X2]…E[Xn] and var(X1 + X2 + … + Xn) = var(X1) + var(X2) + … + var(Xn) Docsity.com Covariance Covariance is a notion that measures the strength and direction of the relationship between two variables. Docsity.com Covariance Covariance is a notion that measures the strength and direction of the relationship between two variables. Experiment: Choose a random UCLA student. •  Let X be his height and Y be his weight. These have positive covariance or are positively correlated. Docsity.com Covariance Covariance is a notion that measures the strength and direction of the relationship between two variables. Experiment: Choose a random UCLA student. •  Let X be his height and Y be his weight. These have positive covariance or are positively correlated. •  Let X be the number of courses he is taking and Y be the average number of hours he sleeps each night. These have negative covariance or are negatively correlated. Docsity.com Covariance Formally, the covariance of two random variables X and Y is defined as cov(X, Y) = E[ (X–E[X]) (Y–E[Y]) ] = E[XY] – E[X]E[Y] Docsity.com Covariance Formally, the covariance of two random variables X and Y is defined as cov(X, Y) = E[ (X–E[X]) (Y–E[Y]) ] = E[XY] – E[X]E[Y] •  What is cov(X, X)? Docsity.com Covariance Formally, the covariance of two random variables X and Y is defined as cov(X, Y) = E[ (X–E[X]) (Y–E[Y]) ] = E[XY] – E[X]E[Y] •  What is cov(X, X)? •  Suppose X and Y are independent. What is cov(X, Y)? Docsity.com Continuous Random Variables What if a random variable X can take on a continuum of different values? •  Exact execution time of a task •  Exact lifetime of a component •  Time between two requests to the Google server PMFs don’t quite make sense anymore… Docsity.com Continuous Random Variables What if a random variable X can take on a continuum of different values? •  Exact execution time of a task •  Exact lifetime of a component •  Time between two requests to the Google server PMFs don’t quite make sense anymore… •  Let X be a random variable whose value is drawn uniformly at random from [0,1]. What is P(X = 0.5)? Docsity.com The Probability Density Function In place of the PMF, we introduce the probability density function (or PDF), denoted fX. For any a and any b ≥ a, € P(a ≤ X ≤ b) = fX (k)dka b ∫ Docsity.com The Probability Density Function For very small values of δ, we can approximate This gives us a nice interpretation of fX(a) as the probability mass per unit length near a Need to be careful with this interpretation though!!! Note that, for example, fX(a) can be bigger than 1… € P(a ≤ X ≤ a +δ) = fX (k)dka a+δ ∫ ≈ fX (a)δ Docsity.com Expectations Expectation and variance of continuous random variables are similar to the discrete case. € E[X] = ??? Docsity.com Expectations Expectation and variance of continuous random variables are similar to the discrete case. € E[X] = kfX (k)dk−∞ ∞ ∫ Docsity.com Expectations Expectation and variance of continuous random variables are similar to the discrete case. € E[X] = kfX (k)dk−∞ ∞ ∫ € E[g(X)] = g(k) fX (k)dk−∞ ∞ ∫ € var(X) = E[(X − E[X])2] = (k − E[X])2 fX (k)dk−∞ ∞ ∫ Docsity.com Uniform PDFs Let X be a random variable whose value is drawn uniformly at random from the range [a, b] Docsity.com Uniform PDFs Let X be a random variable whose value is drawn uniformly at random from the range [a, b] •  What is fX? What is E[X]? What is var(X)? Docsity.com Exponential Random Variables Exponential random variables model the amount of time until an incident of interest takes place •  Length of time before a message arrives at the computer •  Length of time before a light bulb burns out PDF: fX(k) = λe-λk Docsity.com Exponential Random Variables Exponential random variables model the amount of time until an incident of interest takes place •  Length of time before a message arrives at the computer •  Length of time before a light bulb burns out PDF: fX(k) = λe-λk E[X] = λ-1 var(X) = λ-2 These can be verified using integration by parts. Docsity.com Cumulative Distribution Functions A cumulative distribution function (CDF), denoted FX, “accumulates” probability up to a certain value of X FX(k) = P(X ≤ k) Docsity.com Cumulative Distribution Functions A cumulative distribution function (CDF), denoted FX, “accumulates” probability up to a certain value of X FX(k) = P(X ≤ k) •  For discrete random variables, •  For continuous random variables, € FX (k) = pX (x) x≤k ∑ € FX (k) = ??? Docsity.com Cumulative Distribution Functions A cumulative distribution function (CDF), denoted FX, “accumulates” probability up to a certain value of X FX(k) = P(X ≤ k) •  For discrete random variables, •  For continuous random variables, € FX (k) = pX (x) x≤k ∑ € FX (k) = fX (x)dx−∞ k ∫ Docsity.com Cumulative Distribution Functions A cumulative distribution function (CDF), denoted FX, “accumulates” probability up to a certain value of X FX(k) = P(X ≤ k) •  For discrete random variables, •  For continuous random variables, € FX (k) = pX (x) x≤k ∑ € FX (k) = fX (x)dx−∞ k ∫ → fX (k) = dFX dk (k) Docsity.com Cumulative Distribution Functions Consider a uniform continuous random variable X Docsity.com Cumulative Distribution Functions Consider a uniform continuous random variable X Docsity.com Cumulative Distribution Functions Consider a uniform continuous random variable X Note that CDFs are always monotonically non-decreasing Docsity.com