Download Analysis of Multistage Games: Repeated Games & Subgame Perfect Equilibrium and more Study notes Systems Engineering in PDF only on Docsity!
Recap
! Last class (January 20, 2004) ! Duopoly models ! Multistage games with observed actions ! Subgame perfect equilibrium ! Extensive form of a game ! Two-stage prisonerís dilemma ! Today (January 22, 2004) ! Finitely repeated games ! Infinitely repeated games ! Prisonerís dilemma ! Friedmanís Theorem ! Repeated Cournot game
Repeated games
theT stage games.
payoffsforG(T)arethe(discounted)sumofthepayoffsfrom
precedingplaysobserved beforethenextplaybegins.The
in whichGisplayedTtimes,with theoutcomesofall
GivenastagegameG,letG(T)denotethe
the oftherepeatedgame.
receivepayoffs ( ,..., ),..., ( ,..., ).WecallG
choosetheiractions ,..., fromactionspaces ,..., and
completeinformationin whichplayers1,..., simulateneously
LetG { ,..., ; ,..., }denoteastaticgameof
1 1 n 1
1 1
1 1 n
game
finitelyrepeated
stage game
a a a a
a a A A
n
A A
n n
n n
n
Repeated games
! Result : If the stage game G has a unique Nash equilibrium then for any finite T, the repeated game G(T) has a unique subgame-perfect outcome: the Nash equilibrium of G is played in every stage.
Example
R 0, 0 0, 0 3, 3
M 0, 5 4, 4 0, 0
L 1, 1 5, 0 0, 0
L M R
Player 1
Player 2
! The stage game is played twice ! The first-stage outcome is observed before the second stage begins
Observation
! Let G be a static game of complete information with multiple Nash equilibria. There may be subgame-perfect outcomes of the repeated game
G(T) in which for anyt <T, the outcome in staget
is not a Nash equilibrium of G.
Definitions
! In the finitely repeated game G(T), a playerís
strategy specifies the playerís actions in each stage,
for each possible history of play through the previous stages.
! In the finitely repeated game G(T), asubgame
beginning at stage t+1 is the repeated game in which G is played T-t times, denoted by G(T-t).
Example
! All possible outcomes (histories) at the end of stage 1: (L,L) (L,M) (L,R) (M,L) (M,M) (M,R) (R,L) (R,M) (R,R) ! (M, L, L, L, L, R, L, L, L, L) Play M in the first stage; Play L in the second stage unless the first stage outcome is (M,M)
R 0, 0 0, 0 3, 3
M 0, 5 4, 4 0, 0
L 1, 1 5, 0 0, 0
L M R
Player 1
Player 2
Infinitely Repeated Prisonerís Dilemma
! The game is repeated infinitely ! For each t, the outcomes of the previous t-1 stage games are observed ! Payoffs?
D 5, 0 1, 1
C 4, 4 0, 5
C (cooperate) D (defect)
Prisoner 1
Prisoner 2
Average payoffs
! V = π 1 + δπ 2 + δ^2 π 3 + Ö = ∑t=1→∞ δt-1πt ! If we received an ìaverageî payoff of π in every stage, then V = π+ δπ+ δ^2 π+ Ö = π(1+ δ+ δ^2 + Ö )= π/(1- δ) ! π/(1- δ) = ∑t=1→∞ δt-1πt.
π = (1- δ) ∑t=1→∞ δt-1πt.
Example: Payoffs 4 4 4 4 4 Ö. Average payoff = 4 Net present value = 4/ (1- δ)
Infinitely repeated games
sequenceofstage games.
isthepresent valueoftheplayer'spayoffsfromtheinfinite
beforethe stagebegins.Eachplayer'spayoffinG( , )
of the 1 precedingplaysofthestagegameareobserved
playerssharediscountfactor .Foreach ,theoutcomes
in whichGisrepeatedforeverandthe
GivenastagegameG,letG( , )denotethe
th δ
δ
δ
t
t-
t
repeated game
infinitely
Infinitely Repeated Prisonerís Dilemma
! Strategy:
Play C in the first stage. In the t th^ stage, if the outcome of all t-1 preceding stages has been (C,C), then play C; otherwise, play D
D 5, 0 1, 1
C 4, 4 0, 5
C (cooperate) D (defect)
Prisoner 1
Prisoner 2
Definitions
! In an infinitely repeated game G(∞,δ), a playerís
strategy specifies the playerís actions in each stage,
for each possible history of play through the previous stages.
! In the infinitely repeated game G(∞,δ), each
subgame beginning at stage t+1 is is identical to
the original game G(∞,δ).
Trigger strategies for Prisonerís Dilemma
! Two types of subgames:
(i) Subgames where the outcomes of all previous stages have been (C,C) The trigger strategies are Nash equilibrium for this class of subgames, as well as for the original game.
(ii) Subgames where the outcome of at least one earlier stage differs from (C,C) Playerís strategies are to repeat (D,D) forever, which is also a Nash equilibrium for the original game
Observation
! Even if the stage game G has a unique Nash equilibrium, there may be subgame-perfect outcomes of the infinitely repeated game in which no stageís outcome is a Nash equilibrium of G.
Feasible payoffs in the stage game
! The payoffs (π^1 , π^2 , Ö , πn) arefeasible in the stage
game G if they are a convex combination of the pure-strategy payoffs of G.
Example
! What are the pure-strategy payoffs? ! (4,4) (0,5) (5,0) (1,1)
D 5, 0 1, 1
C 4, 4 0, 5
C (cooperate) D (defect)
Prisoner 1
Prisoner 2
Friedmanís Theorem
! Let G be a finite static game of complete information. Let (e^1 , e^2 , Ö , en) denote the payoffs from a Nash equilibrium of G and let (x^1 , x^2 , Ö , xn) denote any other feasible payoffs from G. If xj> ej for every player j and if δ is sufficiently close to 1, then there exists a subgame-perfect Nash equilibrium of the infinitely repeated game G(∞,δ) that achieves (x 1 , x^2 , Ö , xn) as the average payoff.
Feasible payoffs in the Prisonerís Dilemma
Player 1 payoffs
Player 2 payoffs (0,5)
(1,1) (5,0)
(4,4)
Proof of Friedmanís Theorem
! Let (a (^) e1 , ae2 , Ö , aen) be the Nash equilibrium of G that yields the equilibrium payoffs (e^1 , e^2 , Ö , en).
! Let (a (^) x1 , ax2 , Ö , axn) be the collection of actions that yields the equilibrium payoffs (x^1 , x^2 , Ö , xn).
! Trigger strategy for player i: ! Play axi in the first stage. In the tth^ stage, if the outcome of all t-1 preceding stages has been (ax1 , ax2 , Ö , axn) then play a (^) xi; otherwise, play aei.
! Show that the trigger strategies induce a NE
! Show that the equilibrium is subgame perfect
Proof of Friedmanís Theorem (cont.)
! Suppose all players other than player i use the trigger strategy.
! Best response of player i in stage t: ! If the outcome of the previous stage differs from (ax1 , ax2 , Ö , axn) ! Play aei forever ! If the outcomes of all previous stages are (ax1 , ax2 , Ö , axn)
i e en
i x xi xi xi xn
i
x xi i xi xn
i a A
d a a a a a a a e
a a a a a d
i
i i i
− +
∈ − +
max ( ,..., , , ,..., )
1 , 1 , 1 1
1 , 1 , 1
π π
π
Repeated Cournot Game
! Cournot stage game ! Two competing firms, selling a homogeneous good ! Themarginal cost of producing each unit of the good: c ! The market price, P is determined by (inverse) market demand: ! P=a-Q if a>Q, P=0 otherwise. ! Each firm decides on the quantity to sell (market share): q 1 and q (^2) ! Q= q 1 +q 2 total market demand ! Both firms seek to maximize profits ! Unique NE of the stage game: qC=(a-c)/3 Q= 2(a-c)/
! Monopoly quantity: qM=(a-c)/
Repeated Cournot Game (cont.)
! The stage game is repeated infinitely many times
! The firms have discount factor δ
! Trigger strategy ! Produce half the monopoly quantity, qM/2, in the first stage. In the tth^ stage, produce qM/2 if both firms have produced qM/2 in all previous stages; otherwise, produce qC.
! Show that the trigger strategy induces a subgame perfect NE.
Repeated Cournot Game (cont.)
! Profit of one firm ! If both produce qM/2: (a-c)^2 /8 = πM/ ! If both produce qC^ : (a-c) 2 /9 = πC ! Best response of firm i: ! If the last stage outcome is other than (qM/2, qM/2) ! Play qC^ forever ! If all previous stagesí outcomes are (qM/2, qM/2) ! Deviate max (a-qi-qM/2-c) qi → qi = 3(a-c)/8 πD^ = 9(a-c) 2 / V i^ = πD^ + πC^ δ /(1- δ) ! Play qM/ V i^ = πM/2 + δ V i^ → V i^ = πM^ /2(1- δ)
Repeated Cournot Game (cont.)
! Profit of one firm ! If both produce q M/2: (a-c) 2 /8 = πM/ ! If both produce q C^ : (a-c) 2 /9 = πC
! Best response of firm i:
! If the last stage outcome is other than (q M/2, q M/2) ! Play qC^ forever ! If all previous stagesí outcomes are (qM/2, qM/2) ! Deviate: Vi^ = πD^ + πC^ δ /(1- δ) ! Play qM/2: Vi^ = πM^ /2(1- δ) ! Playing the trigger strategy is NE iff πM^ /2(1- δ) ≥ πD^ + πC^ δ /(1- δ) → δ ≥ 9/
Repeated Cournot Game (cont.)
! Best response of firm i: ! If the last stage outcome is other than (q ^ , q^ ) ! Play qC^ forever ! If all previous stagesí outcomes are (q, q ^ ) ! Deviate: Vi^ = πD^ + πC^ δ /(1- δ) ! Play q: Vi^ = π^ + δ Vi^ → Vi^ = π^ /(1- δ) ! Playing the trigger strategy is NE iff π^ /(1- δ) ≥ πD^ + πC^ δ /(1- δ) Substitute and solve for q*^ : q *= (9-5δ)(a-c)/3(9-δ)
Recall: q C=(a-c)/3 qM=(a-c)/
Example: Wage setting
! Stage game
! One firm, one worker ! The firm offers the worker a wage, w ! The worker accepts or rejects the firmís offer ! Reject: the worker becomes self-employed at wage w (^0) ! Accept: Work (disutility e), or Shirk (disutility 0) ! If the worker works (supplies effort): Output is high=y ! If the worker shirks: Output is high with probability p, and low=0 with probability 1-p ! The firm does not observe the workerís effort decision ! The output of the worker is observed by both parties
Example: Wage setting (cont.)
! Payoffs (Firm,Worker)
! Work (Supply effort) ! High output: (y-w,w-e) ! Shirk ! High output: (y-w, w) ! Low output: (-w, w)
! What is the subgame-perfect equilibrium in this stage game? ! For any w ≥ w 0 , worker accepts employment and shirks ! Firm offers w=0 (or any other w<w 0 )
Example: Wage setting (cont.)
! Strategies ! Firm: Offer w=w* in the first stage. In stage t, ! offer w=w* if the history of play ishigh-wage, high- output (all previous offers have been w*, all previous offers have been accepted, and all previous outputs have been high) ! otherwise, offer w= ! Worker: ! If w>w 0 , accept the firmís offer and supply effort if the history of play, including the current offer, is high- wage, high-output (shirk otherwise) ! If w<w 0 , choose self -employment
