Probability of Compound Events: Mutually Exclusive and Independent Events, Slides of Parallel Computing and Programming

Download Probability of Compound Events: Mutually Exclusive and Independent Events and more Slides Parallel Computing and Programming in PDF only on Docsity!

Probability of Compound Events

Basics—Mutually Exclusive Events

• p(A) will denote prob. of event A.
• Two events A, B are mutually exclusive (ME) if one

event happening precludes the possibility of the

other. In other words p(both A and B happens) =

p(A  B) = 0.

• E.g., Event Di is a stranger’s b’day being on the i’th

day of the week, 1 <= i <= 7. p(Di) = 1/7 for each i.

However, given that the event D4 has happened

(the stranger’s b’day is found to be on Thurs), the

prob. of the other events are 0. Thus the Di’s are

ME.

• Another ex. is blocks of code in an if-then-else

chain. The event that a particular block Bj will be

executed in the current pass through the if-then-

else chain is ME w/ the execution of other blocks

• If A and B are ME, then the p(either A or B

happens) = p(A U B) = p(A) + p(B); see Fig. 1

• If A, B are not ME, then p(AUB) = p(A) + p(B) – p(A

 B), since we are counting the event p(A  B)

twice in p(A) + p(B); see Fig. 2.

• Similarly, p(A U B U C) = p(A) + p(B) + p(C) – p(A 

B) – p(B  C) – p(A  C) + p(A  B  C), since we

included p(A  B  C), 3 times in p(A) + …+ p(B)

but also excluded it 3 times in – p(A  B) – p(B 

C) – p(A  C), and so need to add it back once; see

Fig. 3.

• The general formulation for p(Ui=1n^ Ei) is obtained

from a generalization of the above –inclusion-

exclusion pattern and is formally called the

inclusion-exclusion principle ; see

http://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle

Fig. 1

A B

U

Fig. 2

A B

A

B

U

A B

U^ A^ ^ B

C

A  C

B  C

A  B  C

Fig. 3 Docsity.com

Exclusive Atomic Events

• Scenario: There is a box w/ a red ball in it. A group of n randomly

picked individuals are picked and placed equidistant from the box.

At the count of 3, they are to rush to the box and take the ball out.

Clearly, only one person can get the ball. See Figs. 5 and 6.

• The atomic event Ei is person i gets the ball, 1 <= i <= n. p(Ei) = 1/n.

Also, since only one person can get the ball, the Ei’s are pair-wise

mutually exclusive (i.e., ME wrt to each other).

• Hence the prob. that either person i or person j gets the ball p p(Ei

U Ej) = p(Ei) + p(Ej) = 2/n. Similarly, the prob. that any person in a

subset of k people gets the ball = k/n.

• Consider event (Ei)’ = event that i does not get the ball. p((Ei)’) = 1
  • p(Ei) [by definition of the complement event] = (n-1)/n.
• (Ei)’ and (Ej)’ are not ME (both can happen), and they are also not

independent. The latter is so, since if Ei does not get the ball, then

the prob. of Ej increases to 1/(n-1), i.e. p(Ei/(Ej)’) = 1/(n-1) != p(Ei).

Thus p((Ei)’/(Ej)’) = 1 - p(Ei/(Ej)’) = 1 – (1/(n-1)) = (n-2)/(n-1) !=

p((Ei)’).

• Thus p((Ei)’  (Ej)’) = p((Ej)’)p((Ei)’/(Ej)’) = ((n-1)/n)((n-2)/(n-1)) =

(n-2)/n

• Prob. that one of person i or j gets the ball = p(Ei U Ej) can also be

derived as (whether these events are ME or not): 1 – prob.(none of

them get the ball) = 1- p((Ei)’  (Ej)’) = 1 - (n-2)/n = 2/n

• If 2 events A, B are not ME then p(AUB) = 1 –p(A’  B’) is an easier

derivation than determining p(AUB) using the inclusion-exclusion

principle, especially if the number of events in the union is > 2.

• For A, B, C, p(AUBUC) = 1 - p(A’  B’  C’) = 1 – p(A’/ B’  C’) *p(B’ 

C’) = 1 - p(A’/ B’  C’) )p(B’/C’)p(C’).

• For the above scenario, this is 1 – (1-p(A/ B’  C’))*(n-2)/n = 1-(1-

1/(n-2)*(n-3)/n = 1 – (n-3)/n = 3/n

• Thus for the ex. in Fig. 5, p(E1 U E3) can be obtained as either:

p(E1)+p(E3) (as they are ME) = 2/3 or the more general (since ME

atomic events are rare), 1 –p(E1’  E3’) = 1 – p(E1’/E3’)*p(E3’) = 1 –

(1-p(E1/E3’)(1-p(E3)) = 1-(1-1/2)(1-1/3) = 1-(1/2)(2/3) = 2/

p(E1)=1/

p(E2)=1/3 (^) p(E3)=1/

E1 E2 E

ME Events

Fig. 6: Three ME events

Fig. 5: Three persons competing for a single ball.

Exclusive Atomic Events (cont’d)

• Scenario: In an indirect tree topology, the top switch (see Fig. 7) is the sole communication route between the (P/2)-processor subsets on its left and right subtree. We need to know the message load on this switch, when each processor sends a msg. to another random processor. We do this by focusing on a single processor v (=4 in Fig. 7) and determining what the prob. is of its msg. going to the (P/2)-proc. subset on the other side of the tree.
• The atomic events are {Mi: proc. i gets the msg. from v}, w/ only Mv = φ, since v does not send a msg. to itself. For other Mi’s, q = p(Mi) = 1/p-1 (=1/5 in Fig. 7), as the msg. is sent to a random dest. & thus the prob. distr. is uniform.
• The events Mi are also clearly ME, as there is only 1 msg. and only 1 proc. will get it.
• Thus p(M1 U M2) = p(M1)+p(M2) = 2/(p-1) (=2/5 in ex.). In general, p(a proc. in the other (P/2)-proc. subset getting the msg. from v) = p(M1 U M2 … U M (^) P/2 ) = p(M1)+p(M2)+…p(M (^) P/2 ) = (p/2)*q (= 3/5 for the Fig. 7 ex.)
• Similarly, p(a proc. in the other (P/4)-subset within v’s (P/2)-subset getting the msg.) = (p/4)*q.
• Also, as in the prev. “ball-grabbing” scenario, Mi’s are not independent, buy similarly, p(M1 U M2 … U M (^) P/2 ) can also be derived as 1 – p(M1’  M2’  ….  M’P/2 ) = 1 – (p(M1’)p(M2’/M1’)p(M3’/(M1’  M2’))* …. p(M’P/2 /(M1’  M2’ …. M’(P-1)/2 )) = 1 – [(p-2)/(p-1)][(p-3)/(p-2)]…..[(p-p/2 - 1)/(p-p/2)] = 1 – (p/2 – 1)/(p-1) = (p/2)/(p-1) = (p/2)*q.
• Thus the msg load (or prob.) on the top switch in a random msg. pattern is double that of the switch below it, and so on, leading to the rationale for a fat-tree topology in which the switch size and # of links doubles (or increases by a factor > 1 and <= 2) so that msg. latency is not high due to contention/collision.

M1 M2 M ME Events

Fig. 8: Five ME events

Fig. 7: Scenario in which proc. 4 sends a single message to a random destination. Q: What is the prob. of the msg. going through the top switch?

p(Mi) = 1/(P-1) =1/

Top switch in an indirect tree

P/2 processors P/2 processors

M4 M

Non-ME Atomic Events (contd.)

• Event (Mi,j)’ = i does not get a msg. from j. Any two events (Mi,j)’ and (Mi,k)’ , j != k, are independent as the occurrence of one does not affect that of the other. Thus p((Mi)’) = p ( (^) j in right (Mi,j)’) = Π (^) j in right p((Mi,j)’) = Π (^) j in right (1- (1/(P-1)) = Π (^) j in right (P-2)/(P-1) = [(P-2)/(P-1)] P/
• However, (Mi)’ and (Mk)’, i != k, are not independent: no msgs. recvd at i (i.e., (Mi)’ occurring) increases Mk, since it increases each component Mk,j to (1/p-2), and thus decreases (Mk)’.
• Thus p(Mi’  Mk’) = p(Mi’)*p(Mk’/Mi’)
• p(Mk’/Mi’) = p ( (^) j in right (Mk,j)’/Mi’). Each p(Mk,j)’/Mi’) is indep. across changing j’s.
• Thus p ( (^) j in right (Mk,j)’/Mi’) = Π (^) j in right p((Mk,j)’/Mi’) = Π (^) j in right (1- (1/(P-2)) = Π (^) j in right (P-3)/(P-2) = [(P-3)/(P-2)] P/
• Thus p(Mi’  Mk’) = p(Mi’)*p(Mk’/Mi’) = [(P-2)/(P-1)] P/2^ * [(P-3)/(P- 2)] P/2^ = [(P-3)/(P-1)] P/
• Extrapolating in an obvious way, p(i in left ( (^) j in right (Mi,j)’) ) = p(i in left Mi’) = [(P-1 – (P/2))/(P-1)] P/2^ = [(P/2)-1)/(P-1)] P/
• p(Ui in left, j in right Mi,j) = 1 – p(i in left, j in right (Mi,j)’) = 1 – p(i in left ( (^) j in

right (Mi,j)’) ) = 1 - [(P/2)-1)/(P-1)]^ P/^.

• For the ex. in Fig. 9 this prob. = 1 – (2/5)^3 = (125 – 8)/125 = 117/125 = 0..
• For Fig. 10 it is 1 – (3/7)^4 = (2401 – 81)/2401 = 2320/2401 = 0.
• Similarly, p(a proc. in one of the (P/4)-subset within the left (P/2)- subset getting at least msg. from its other (P/4)-subset) [this will be via its 2nd^ topmost switch = 1 - [(3P/4)-1)/(P-1)] P/4^ = (for Fig. 10) 1 – (5/7)^2 = 24/49 = 0.49 (almost ½ of that for the top switch). Thus again, load (in terms of prob. of >= 1 msg. through it) on the top switch is about 2X that of switch below it and so forth

Fig. 10: Scenario in which processors in the left (P/2)-proc. subset sends one message each to a random destination. Q: What is the prob. of at least msg. going through the top switch? This prob. is representative of the msg. load on the top switch under this scenario.

p(Mi,j) = 1/(P-1) =1/

Top switch in an indirect tree

P/2 processors

P/2 processors

Non-ME Atomic Events (contd.)

• Via union and incl-excl principle p(at least 1 msg. to left from right for Fig. 9 ) = (see Fig. 11) p(M1)+p(M2)+p(M3) – p(M1  M2) – p(M1  M3) – p(M2  M3) + p(M1  M2  M3). This will be very involved:
• Each component is quite involved, e.g., p(M1) = 1 – p(M1’) (which itself is using the compl. of compl. approach) = 1 - [(P-2)/(P-1)]P/2^ = (for Fig. 9) = 1 – (4/5)^3 = (125 – 64)/125 = 61/125 = p(M2) = p(M3)
• p(M1  M2) = p(M1) - p(M1  M2’) ; p(M1  M2’) = p(M2’) – p(M1’  M2’)
• From earlier, p(M1’  M2’) = [(P-3)/(P-1)] P/2^ = (3/5)^3 = 27/
• Thus p(M1  M2’) = p(M2’) – [(P-3)/(P-1)] P/2^ = [(P-2)/(P-1)] P/2^ - [(P- 3)/(P-1)] P/2^ = (4/5)^3 - (3/5)^3 = (64-27)/125 = 37/
• p(M1  M2) = p(M1) - p(M1  M2’) = 61/125 – 37/125 = 24/
• p(M1  M2  M3) = p(M1) - p(M1  (M2  M3)’)
• p(M1  (M2  M3)’) = p(M1  (M2’ U M3’)) = p((M1  M2’) U (M1  M3’)) = p(M1  M2’) + p(M1  M3’) - p(M1  M2’  M3’)
• p(M1  M2’  M3’) = p(M2’  M3’)*p(M1/(M2’  M3’))
• p(M1/(M2’  M3’)) = 1 - p(M1’/(M2’  M3’)) = 1 – [1- (1/(P-3))] P/2^ = 1 – [(P-4)/(P-3)] P/2^ = 1 – (2/3)3 = (27-8)/27 = 19/
•  p(M1  M2’  M3’) = p(M2’  M3’)p(M1/(M2’  M3’)) = (27/125)(19/27)
•  p(M1  (M2  M3)’) = p(M1  M2’) + p(M1  M3’) - p(M1  M2’  M3’) = 37/125 + 37/125 - (27/125)(19/27) = 74/125 - (27/125)(19/27)
•  p(M1  M2  M3) = p(M1) - p(M1  (M2  M3)’) = 61/125 – 74/125 + (27/125)*(19/27) = 0.152 – 13/125 = 0.
• Thus p(M1UM2UM3) = p(M1)+p(M2)+p(M3) – p(M1  M2) – p(M1  M3) – p(M2  M3) + p(M1  M2  M3) = 3* 61/125 - 3* 24/125 - 0.048 = 3*37/125 - 0.048 = 0.888 + 0.048 = 0.936 (same as in prev. slide)

M1 M

U^ M1^ ^ M

M

M1  M

M2  M

M1  M2  M

Fig. 11 Non-ME events Mi for left processors 1-3 for scenario of Fig. 9

Fig. 9: Scenario in which processors in the left (P/2)-proc. subset sends one message each to a random destination. Q: What is the prob. of at least msg. going through the top switch? This prob. is representative of the msg. load on the top switch under this scenario.

p(Mi,j) = 1/(P-1) =1/

Top switch in an indirect tree

P/2 processors P/2 processors