Probability and Statistics: Expected Values, Random Variables, and Independence, Study notes of Number Theory

Download Probability and Statistics: Expected Values, Random Variables, and Independence and more Study notes Number Theory in PDF only on Docsity! Y Probability and Expectation; Counting with Repetitions. Zeph Grunschlag L19 2 Agenda More probability  Random Variables  Independence  Expectation More counting  Allowing repetitions  Stars and Bars  Counting solutions to integer inequalities L19 5 Outcomes with Variable Likelihoods In the previous definition of probability, p (E ) = |E | / |S | assumed that all outcomes were equally likely. Sometimes can’t assume this. EG: S = {wait 2 minutes, wait 10 minutes} First outcome was 3 times as likely as 2nd. New assumption for set of outcomes S:  Each outcome s occurs with probability p (s)  0  p (s)  1  Sum over S of the probabilities equals 1 L19 6 Random Variables The definition of random variables seems to involve neither randomness nor variables. DEF: Let S be a finite sample space. A random variable X is a real function X : S  R EG: In terms of previous subway example, can express the amount of time gained under the possible transfer scenarios using random variable: X : { 2min’s, 10min’s }  R X (2min’s) = 3, X (10min’s) = -5 L19 7 Definition of Expectation DEF: Let X be a random variable on the finite sample space S. The expected value (or mean) of X is the weighted average: EG: Consider the NYS lottery. Assume: ticket costs $0.50 jackpot is $18,000,000 no taxes, no inflation, no shared winnings consider only first prize Q: What is the expected net winnings?    Ss sXspX )()()(E L19 10 Expectation of NYS Lottery Detailed Analysis Expected net winnings. Negative 3.6 cents: (18,000,000.00 - 0.50) · 0.0000000222 + (184,854.00 - 0.50) · 0.0000001332 + (2,225.00 - 0.50) · 0.0000070577 + (31.00 - 0.50) · 0.0004587474 + (1.00 - 0.50) · 0.0103982749 + -0.50 · 0.9891357646 = -0.0355 Q: What BIG factor did we forget? L19 11 Expectation of NYS Lottery Bernoulli Trials A: Forgot about possibility of sharing the jackpot! Go back to earlier analysis involving only first prize. Suppose n additional tickets were sold. We need to figure out the probability that k of these were winners –call this probability qk . Jackpot winnings split equally among all winners so expected win value is: X (win) =-0.50+18,000,000(q0/1+q1/2+q2/3 +…) Need a way of computing qk ! L19 12 Bernoulli Trials A Bernoulli trial is an experiment, like flipping coins, where there are two possible outcomes, except that the probabilities of the two outcomes could be different. In our case, the two outcomes are winning the jackpot or not winning the jackpot and each has its own probability. L19 15 Bernoulli Trials A: (1/2)10 · (1/2)10 ·C (20,10) = 184756 / 220 = 184756 / 1048576 = 0.1762… L19 16 Expectation of NYS Lottery Bernoulli Trials Apply formula to NY Lotto: qk = p k · (1-p) n-k ·C (n,k ) = 0.00000002219k ·0.99999997781n-k ·C (n,k ) Assume that n = 11,800,000 L19 17 NYS Lottery Best Expectation Calculation Used these figures to compute qk: Values become negligible for higher k. Plug into X (win)=-0.50+18,000,000(q0/1+q1/2+q2/3+…) =-0.50 + 18,000,000 · 0.8798  $15,836,000. Plugging this back in to above, the most accurate approximation for expected winning is: -7.9¢ k 0 1 2 3 4 qk 0.77 0.20 0.026 0.0023 0.00015 L19 20 Sum Rule for Expectations THM: Suppose X1, X2, …, Xn are random variables over the same sample space. Then: E(X1+X2+…+Xn ) = E(X1)+ E(X2 )+…+E(Xn ) Proof : RHS )()()()()()( )()]()()([LHS 21 21        spsXspsXspsX spsXsXsX Ss n SsSs n Ss   L19 21 Sum Rule for Expectations EG: Find the expected number heads when n coins are tossed. Let X be the random variable counting the number of heads in a sequence of n tosses. For example, if n = 3, X(HTH) = 2, X(TTT)=0. We can break X up into a sum X = X1+X2+…+Xn where Xi = 1 if i th toss comes up H and 0 if T. Therefore: E(X ) = E(X1)+ E(X2 )+…+E(Xn ) By symmetry, E(X1)=E(X2 )=…=E(Xn ) so E(X ) = n ·E(X1). Q: What is E(X1)? L19 22 Sum Rule for Expectations A: E(X1) = ½. (As a probability E(X1) is just the likelihood that the first head will be a head.) Plugging back in: E(X ) = n ·E(X1) = n / 2 which means that when n coins are tossed, we expect half to come up heads! L19 25 Conditional Probability EG: What is the probability that a length 4 bit string contains 00 given that they start with 1? E = {contain 00} F = {starts with 1} EF = {1000,1001,1100} p (E\F ) = |EF | / |F | = 3/23 DEF: If E and F are events and p (F ) > 0 then the conditional probability of E given F is defined by: p (E |F ) = p (EF ) / p (F ) L19 26 Independence An event E is said to depend on an event F if knowing that F occurs changes the probability that E occurs. EG: Rain is much likelier on a cloudy day than in general so E and F are dependent. Conversely, E is independent of F if p (E\F ) = p (E ). In other words: p (EF )/p (F ) = p (E ); equivalently: p (EF ) = p (E ) · p (F ) L19 27 Independence Q: In length 4 bit strings. Is containing 00 independent from starting with 1? E = {contain 00} F = {starts with 1} EF = {1000,1001,1100} L19 30 Independence of Random Variables Random variables are defined to be independent if the probabilities that they will take on any particular values is independent: DEF: The random variables X and Y are independent if for all values x,y the event “X=x” is independent from the event “Y=y”. Q: Is the value of a cast die independent from the event of casting a 2? L19 31 Independence of Random Variables A: NO! Intuitively, if we know that a cast die comes up “2”, then the value of the die is forced to be 2, so there can’t be independence. Formally: Sample space: S ={1,2,3,4,5,6} Rand. var. for die-value: X (i )=i Rand. var. for casting a 2: Y(j ) = 1 if j = 2, and Y(j ) = 0 otherwise. Set x = 2, y = 1 we have p(X=x) = 1/6. p(Y=y) = 1/6. But p(X=x and Y=y) = 1/6 which is not equal to p(X=x)·p(Y=y) = 1/6·1/6 = 1/36. L19 32 Variance and Standard Deviation In reporting midterm score I mentioned mean and standard deviation. Formally, given n students we set up a random variable X which inputs a student and outputs the score of the students. The mean is just the expectation: m = E(X ) = 66.1 The variance measures how far scores were in general from the expected: v = E( (X-m)2 ) = 419.471 The standard deviation is the root mean square (RMS) of the difference from the expected, i.e. the square-root of the variance: s = v = 20.481 L19 35 Integer Linear Programming It turns out the the following algorithmic problem is very important to computer science. In fact, almost every algorithmic problem can be converted to this problem as it is “NP-complete”. Integer Linear Programming: Given integer variable inequalities with integer coefficients, find a solution to all variables simultaneously which maximizes some function. EG: Find integers x,y,z satisfying: x  0, y  0, z  0, x+y+z  136, x+y+z  136 and maximizing f (x) = 36x - 14y + 17z L19 36 Integer Linear Programming Unfortunately, there is no known fast algorithm for solving this problem. In general, forced to try every possibility and keep track of (x,y,z) with current best f (x,y,z). Would like to get an idea at least, of how many non-negative integer solutions there are to x+y+z = 136 before commencing search for best (x,y,z) so have idea of how long solution will take to find. L19 37 Stars and Bars Counting with Repetitions EG: To find the number of non-negative integers solutions to x+y+z = 136 convert to: Given 136 ’s, how many ways are there to break these up into 3 piles (x-pile, y-pile and z- pile)? This is just the number of way that to |’s can be dropped within the 136 stars:  … | … | …  x-pile y-pile z-pile L19 40 Stars and Bars A: How many ways are there to buy 13 bagels from 17 types? Let xi = no. of bagels bought of type i. Interested in counting the number of solutions to x1+x2+…+x17 = 13. Therefore, answer is C (16+13,13) = C (29,13) = 67,863,915. Q: How many solutions in N are there to x1+x2+x3+x4+x5 = 21 if x1≥ 1 ? L19 41 Stars and Bars A: x1+x2+x3+x4+x5 = 21 & x1≥ 1 : |{x1+x2+x3+x4+x5 = 21 | x1≥ 1 } | = |{x1+x2+x3+x4+x5 = 20} | This is because one  is forced to be on pile 1, so are asking how many ways are there to distribute remaining 20 ’s. Answer = C (24,4) = 10,626 Q: How many solutions in N are there to x1+x2+x3+x4+x5 = 21 if x1≥ 2 , x2≥ 2 , x3≥ 2 , x4≥ 2 and x5≥ 2 ? L19 42 Stars and Bars A: x1+x2+x3+x4+x5 = 21, x1≥2, x2≥2 , x3 ≥2, x4 ≥ 2 and x5 ≥ 2 : Same idea. 2 ’s are forced to remain on each of 5 piles. So this is the same as counting solutions of x1+x2+x3+x4+x5 = 11. So answer is C (15,4) = 1365 Q: How many solutions in N are there to x1+x2+x3+x4+x5 = 21 if x1 < 11 ?