Download Random Variables - Essay - Mathematics and more Essays (high school) Mathematics in PDF only on Docsity!
Notes for Chapter 3 of DeGroot and Schervish
Random Variables
In many situations we are not concerned directly with the outcome of an experiment, but instead with some function of the outcome. For example when rolling two dice we are generally not interested in the separate values of the dice, but instead we are concerned with the sum of the two dice.
Definition: A random variable is a function from the sample space into the real numbers.
A random variable that can take on at most a countable number of possible values is said to be discrete.
Ex. Flip two coins.
S = {TT, HT, TH, HH} Let X = number of heads Outcomes Value of X TT 0 HT 1 TH 1 HH 2
X is a random variable that takes values on 0,1,2.
We assign probabilities to random variables.
P(X=0) = 1/ P(X=1) = 1/ P(X=2) = 1/
Draw pmf
All possible outcomes should be covered by the random variable, hence the sum should add to one.
Note that you could define any number of random variables on an experiment.
Ex. In a game, which consists of flipping two coins, you start with 1$. On each flip, if you get a H you double your current fortune, while you lose everything if you get a T.
X = your total fortune after two flips
Outcomes Value of X TT 0 HT 0 TH 0 HH 4
X is a random variable that takes values on 0 and 4.
P(X=0) = 3/ P(X=4) = 1/
For a discrete random variable we define the probability mass function p(x) of X, as p(x) = P(X=x).
Note that random variables are often capitalised, while their values are lower-case.
If a discrete random variable X assumes the values x1, x2, x3, … then (i) p(xi) >= 0 for i=1,2,3..... (ii) p(x) = 0 for all other values of x. (iii) sum_{i=1}^{/inf} p(xi) = 1
The cumulative distribution function (c.d.f) F of the random variable X, is defined for all real numbers b, by F(b) = P(X Ex. Independent trials that consist of flipping a coin that has probability p of turning up heads, is performed until a head appears. Let X= number of times the coin is flipped. What is the c.d.f of X?
X is a discrete random variable that can take on values of 1, 2, 3, 4,.....
H, TH, TTH, TTTH, TTTTH, TTTTTH, TTTTTTH, ........
P(X=1) = p P(X=2) = (1-p) p P(X=3) = (1-p)^2 p ....... P(X=i) = (1-p)i-^1 p .......
For b=1,2,3, ..... we have F(b) = P(X≤ b) = sum_{i=1}^b P(X=i) = sum_{i=1}^b p(1-p)i-^1 = sum_{j=0}^(b-1) p(1-p)j = p(1-(1-p)b)/(1-(1-p)) = 1-(1-p)b, so F(y) = 0 for y< (1-(1-p) b) for b≤ y<(b+1)
It is important to note that all probability questions about a random variable X can be answered in terms of the pmf or cdf: these are equivalent, and both contain all the information we need.
For example we may like to calculate P(a Continuous Random Variables
A random variable is a function from the sample space into the real numbers. A random variable that can take on at most a countable number of possible values is said to be discrete.
Ex. Flip a coin four times.
X = number of heads
p(k) = P(X=k) = (4 k) (1/2)^k (1/2)^(4-k)
p(0) = 1/ p(1) = 4/ p(2) = 6/ p(3) = 4/ p(4) = 1/
Draw pmf
0
! = + + + +^ =
i =
p i
Total Area = 1
P ( 1! X! 3 )= p ( 1 )+ p ( 2 )+ p ( 3 )= + + = =
7/8 of the total area is the area from 1 to 3.
Often there are random variables of interest whose possible values are not countable.
Ex. Let X = the lifetime of a battery. X takes values: 0 ≤x<∞ (all non-negative real numbers)
A random variable X is called continuous if there exists a nonnegative function f, defined for all real x ∈ (-∞, ∞), such that for any set B of real numbers
B
P ( X B ) f ( x ) dx.
The function f is called the probability density function of X. Sometimes written fX.
Compare with the probability mass function. All probability statements about X can be answered in terms of the pdf f.
b
a
P ( a X b ) f ( x ) dx.
!
"!
P X f x dx
The density can be larger than one (in fact, it can be unbounded) – it just has to integrate to one. C.f. the pmf, which has to be less than one.
An interesting fact: ( = )=! ( ) = 0
a
a
P X a f x dx.
The probability that a continuous random variable will assume any fixed value is zero. There are an uncountable number of values that X can take, this number is so large that the probability of X taking any particular value cannot exceed zero.
This leads to the relationship P ( X < x )= P ( X! x ).
The cumulative distribution function of X is defined by
#"
x F ( x ) P ( X x ) f ( y ) dy^.
As before we can express probabilities in terms of the cdf:
P ( a X b ) f ( x ) dx f ( x ) dx f ( x ) dx F ( b ) F ( a ).
b b a
a
!$ !$
The relationship between the pdf and the cdf is expressed by
#"
x F ( x ) f ( y ) dy
Differentiate both sides with respect to x: F ( x ) f ( x ) dx
d =.
Uniform Random Variables
A random variable is said to be uniformly distributed over the interval (0,1) if its pdf is given by
= #^ < <
0 otherwise
x f x.
This is a valid density function, since f ( x )! 0 and ( ) 1
1
0
! =^!^ =
"
"
f xdx dx.
The cdf is given by
F x f ydy dy x
x x
#" 0
( ) ( ) for x in (0,1).
1 x 1
0 x 1
0 x 0 F ( x ) x
Draw pdf and cdf.
X is just as likely to take any value between 0 and 1.
For any 0 Jointly Distributed Random Variables
So far we have dealt with one random variable at a time. Often it is interesting to work with several random variables at once.
Definition: An n-dimensional random vector is a function from a sample space S into Rn.
For a two-dimensional (or bivariate) random vector, each point in the sample space is associated with a point on the plane.
Ex. Roll two dice. Let X = sum of the dice Y= the absolute value of the difference of the two dice.
S={(i,j) | i,j=1,….6} S consists of 36 points. X takes values from 2 to 12. Y takes values between 0 and 5.
For the sample point: (1,1), X=2 and Y= (1,2), X=3 and Y=1 etc….
The random vector above is an example of a discrete random vector.
Definition: If (X, Y) is a random vector then the joint cumulative distribution function FXY of (X, Y) is defined by FXY ( a , b )= P ( X! a , Y! b ) # !" a , b "!.
The function FXY has the following properties:
F (^) XY ( ", ")= a #lim", b #" P ( X! a , Y! b ) = 1
F (^) XY ( "#, b )= a lim$"# P ( X! a , Y! b )= 0
F (^) XY ( a ,"# )= b lim$"# P ( X! a , Y! b )= 0 F (^) XY ( a ,") =lim b #" P ( X! a , Y! b )= P ( X! a )= FX ( a ) F (^) XY ( ", b )=lim a #" P ( X! a , Y! b )= P ( Y! b )= FY ( b )
FX and FY are called the marginal distributions of X and Y respectively.
All joint probability statements about X and Y can be answered in terms of their joint distribution function.
For a1 Definition: If X and Y are both discrete random variables, we define the joint probability mass function of X and Y by pXY ( x , y )= P ( X = x , Y = y ).
The probability mass function of X can be obtained from pXY(x,y) by
>
:(,) 0
ypxy
p X x P X x px y.
Similarly, we can obtain the pmf of Y by
>
:(,) 0
xpxy
p Y y PY y px y.
Ex. Suppose two balls are chosen from a box containing 3 white, 2 red and 5 blue balls. Let X= the number of white balls chosen Y= the number of blue balls chosen Find the joint pmf of X and Y.
Both X and Y takes values between 0 and 2.
pXY(0,0) =C(2,2)/C(10,2)=1/ pXY(0,1) = C(2,1)C(5,1)/C(10,2)=10/ pXY(0,2) =C(5,2)/C(8,2)=10/ pXY(1,0) = C(2,1)C(3,1)/C(10,2) = 6/ pXY(1,1) =C(5,1)C(3,1)/C(10,2)=15/ pXY(1,2) = 0 pXY(2,0) = C(3,2)/C(10,2)=3/ pXY(2,1) = 0 pXY(2,2) = 0
( , )^110106153
,:(,) 0
! = + + + =^ + =^ =
xypxy >
p x y.
Find the pmf of X.
( 0 ) ( 0 ) ( , )^11010
:( 0 ,) 0
= = =! = +^ + =
yp y >
p X P X px y
:( 0 ,) 0
yp y >
p X P X px y
:( 0 ,) 0
yp y >
p X P X px y
Ex. The joint density function of X and Y is given by
% +
0 otherwise
e (^ ) x y f x y
x y
Find the density function of the random variable X/Y.
0
( 1 )
0
0 0
( ) ( )
" =^!
! +^ ) !
) !!
) ! + (
! +
a a
e e e dy e
a PX aY e dxdy e dxdy Y
X
F a P
y ay y a^ y
ay x y x ay
x y X Y
Hence, 2 ( 1 )
a
F a da
d f a X Y XY , 0 Independent Random Variables
Definition: Two random variables X and Y are said to be independent if for any two sets of real numbers A and B, P ( X! A , Y! B )= P ( X! A ) P ( Y! B ).
If X and Y are discrete, then they are independent if and only if p (^) XY ( x , y )= pX ( x ) pY ( y ), where pX and pY are the marginal pmf’s of X and Y respectively.
Similarly, if X and Y are continuous, they are independent if and only if f (^) XY ( x , y )= fX ( x ) fY ( y ).
Ex. A man and woman decide to meet at a certain location. If each person independently arrives at a time uniformly distributed between 2 and 3 PM, find the probability that the man arrives at least 10 minutes before the woman.
X = # minutes past 2 the man arrives Y = # minutes past 2 the woman arrives
X and Y are independent random variables each uniformly distributed over (0,60).
( 10 ) ( , ) ( ) ( )^1
60
10
(^602)
10
2
60
10
10
0
2
10 10
" =!!! =^ =
=^ -
+^ *
+ < = = =^ -
!
y
y y dy
P X Y f x ydxdy f x f ydxdy dxdy
y
x y
X Y x y
If you are given the joint pdf of the random variables X and Y, you can determine whether or not they are independent by calculating the marginal pdfs of X and Y and determining whether or not the relationship f (^) XY ( x , y )= fX ( x ) fY ( y )holds.
Some notation: indicator function: !
0 otherwise
a x b I a x b
Ex. The joint pdf of X and Y is !
% %
0 otherwise
e^2 e^3 x y f x y
x y (^) XY.
Are X and Y independent?
f (^) XY ( x , y )= 6 e "^2 xe "^3 yI ( 0 < x ) I ( 0 < y <!)= 6 e "^2 xI ( 0 < x <!) e "^3 yI ( 0 < y <!)= h ( x ) g ( y )
X and Y are independent.
Ex. The joint pdf of X and Y is !
0 otherwise
2 4xy 0 1 , 0 1 , 0 1 ( , )
x y x y f (^) XY x y.
Are X and Y independent?
f (^) XY ( x , y )= 24 xyI ( 0 < x < 1 ) I ( 0 < y < 1 ) I ( 0 < x + y < 1 )
X and Y are not independent.
Sums of Independent Random Variables
It is often important to calculate the distribution of X+Y when X and Y are independent.
Theorem: If X and Y are independent continuous random variables with pdfs fX(x) and
fY(y) then the pdf of Z=X+Y is!
"
"
f (^) Z ( z )= fX ( z # y ) fY ( y ) dy.
Proof:
FZ ( z ) = P ( Z " z ) = P ( X + Y " z ) = fX + Y ( x , y ) dxdy x + y " z
## =^ fX ( x )^ fY ( y ) dxdy
x + y " z
= fX ( x ) dxfy ( y ) dy $%
z $ y
$%
%
# =^ FX ( z^ $^ y )^ fY ( y ) dy
$%
%
fz ( z ) = d dz
FX ( z " y ) fy ( y ) dy "#
$ =^
d dz
FX ( z " y ) fY ( y ) dy "#
$ =^ fX ( z^ "^ y )^ fY ( y ) dy
"#
Convolution: denote f_z = f_x * f_y
Ex: X and Y are independent uniform r.v.’s on [0,1]. What is the distribution of Z=X+Y?
fZ ( z ) = fX ( z " y ) fY ( y ) dy "#
$ =^1 (^0 <^ z^ "^ y^ <^1 )^1 (^0 <^ y^ <^1 ) dy
"#
$ =^1 (^0 <^ z^ "^ y^ <^1 ) dy
0
1
$ =^1 "^ |^ z^ "^1 |
Note that we started with two discontinuous functions f_x and f_y and ended up with a continuous function f_z. In general, convolution is a smoothing operation – if h=f*g, then h is at least as differentiable as f or g.
Sum of independent integer-valued random variables Theorem: If X and Y are independent integer-valued random variables with pmfs pX(x)
and pY(y) then the pmf of Z=X+Y is = ! "
k
p (^) Z ( z ) pX ( k ) p Y ( z k ).
Another way of calculating the pdf of Z is by using Moment Generating Functions. We will discuss this more later.
