Game Theoretic Approach to Traffic Flow Control: A Multi-Agent Systems Perspective, Exercises of Game Theory

Download Game Theoretic Approach to Traffic Flow Control: A Multi-Agent Systems Perspective and more Exercises Game Theory in PDF only on Docsity!

A Game Theoretic Approach to Traffic Flow Control

(Multi-Agent Systems: Paper Project)

May 2004

Authors Jin Yu Enrico Faldini

Programme Master in Artificial Intelligence Master in Artificial Intelligence

Application Modeling Theoretical Analysis

Work Partition Although we get our work partitioned, we both take part in each other’s

work. And we worked together on the selection of the application and the approach evaluation.

Table of Contents

1 Introduction ...................................................................................................................

2 Game Theory .................................................................................................................

3 Traffic Flow Control......................................................................................................

3.1 Road Users as Autonomous Agents ....................................................................

3.2 Traffic Authority as Central Manager .................................................................

4 Game Theoretic Modeling of the Traffic Problem ........................................................

4.1 Drivers as Game Players .....................................................................................

4.1 Model of the Transportation Network.................................................................

4.3 Nash Equilibrium and Optimal Assignments......................................................

4.3 Congestion Pricing Mechanism ........................................................................

5 Discussion and Conclusions ........................................................................................

References: .........................................................................................................................

in urban roads. An extended study of traffic flow theory as well as the exploration on driver behavior is carried out to analyze the reasons behind congestion. In this paper we will make an attempt to model a simplified subset of the general phenomena of urban traffic that we suppose can be described using some of the tools provided by notions imported from Game theory and the MAS approach.

2 Game Theory

Game theory is the formal study of conflict and cooperation. Game theoretic concepts apply whenever the actions of several players are interdependent. These may be individuals, groups, firms, or any combination of these. The concepts of game theory provide a language to formulate structure, analyze, and understand strategic scenarios. The object of study of game theory is strategic games (games in which the outcome of an action is partially controlled by the players), which is a formal model of an interactive situation. The formal definition lays out the players, their preferences, and the strategic actions available to them, and how these influence the outcome. There is a central assumption in game theory. That is: the players are rational. A rational player is one who always chooses an action which gives the outcome he most prefers, given what he expects his opponents to do.

In Game theory one can always start to think about a particular problem, by asking what would be the best strategy for the players involved (what would be their most rational actions). Perhaps the most important notion in the theory of rational behavior in games is the Nash equilibrium. A Nash equilibrium is an action profile with the property that no player can do better by deviating from this profile, given that every other player adheres to it. We cannot claim to be able to find out all the criteria for assigning what are the most rational choices in all possible games. But in order to understand a critical issue that we will address later, we would like to analyze in some detail the famous Prisoner’s dilemma.

In this kind of game, a player has two options (two strategies): he either confesses his crime (a) or not (b). Generally, if one of the strategies always results in a better outcome for the player, we can say that such strategy strongly dominates the other, no matter what would be the choice of the other player. Normally, if we can find such a strategy in a game, we are tempted to use it every time. As it is well-known in the prisoner’s dilemma, defecting (confessing the crime) should lead to a better outcome in this case. But, interestingly, there is more to this game than choosing a

dominating strategy whenever possible. The thing is: if both players choose to play that move, they will have an outcome that is not so good as if they had chosen not to confess (cooperating in this way with each other). This is an interesting result that translates very well the intuition that even selfish agents have to at least partially cooperate with each other to achieve a better payoff of their actions.

Another concept from GT helps us to understand the issue of why the interaction of selfish rational agents can lead to situations in which the final result is undesirable. Pareto efficiency speaks about the global result for all players in a game: if an outcome is Pareto efficient there is no other outcome that makes a player better off without making another player worse off. In the case of the prisoners’ dilemma, if both players choose to defect, the final outcome is not Pareto efficient. The important thing to bear in mind at this point is that Nash equilibrium does not guarantee Pareto efficiency. Even if there are situations where local interactions are sufficient to guarantee a global intelligent order, what we are just about to see is that, sometimes, local interactions can lead to some organization (in a Nash equilibrium form) that is sub optimal, or that is Pareto inefficient.

3 Traffic Flow Control

Traffic flow control is a set of traffic engineering strategies which are used to regulate the traffic flow in urban areas. Management of traffic flow is expected to create a traffic pattern which increases the efficiency of public transport. Possible approaches to control traffic flows are traffic light signaling, dynamic route information panels, radio broadcast messages, etc. In this paper, we will focus on the traffic flow control scenario where there are two parties involved, namely the road users (i.e. car drivers) who are supposed to travel according to his or her best judgment of the traffic stream's speed; and the traffic authority who as the central manager is responsible for regulating the traffic flow.

3.1 Road Users as Autonomous Agents

In the traffic scenario mentioned above, the major party is formed by road users (or more specifically, car drivers). As autonomous entities in the traffic network they can be characterized by the following aspects:

driving speed and charging for the use of roads etc. Thus, by introducing rules and regulations into the existing traffic system, the traffic authority regulates the behavior of self-interested car drivers indirectly so that the quality of the whole traffic system is improved.

4 Game Theoretic Modeling of the Traffic Problem

As discussed above, drivers are assumed to act selfishly and spontaneously to minimize their own travel cost/time, while they are indifferent to the “social optimum”, such as minimum total latency of the transportation network. This independent and non-cooperative behavior of drivers obviously evokes game theory and its main concept of rational behavior, the Nash equilibrium (or Wardrop equilibrium [1], which is defined in the context of transportation network). In the following sections, we will discuss the traffic control problem from the game theoretic perspective.

4.1 Drivers as Game Players

Game theory is concerned with strategic games, in which two or more players participate in a cooperative/non-cooperative task. A strategic game can be generally characterized by its players, their strategies and pay-offs. To draw a parallel in the context of the transportation network, we can think of the traffic flows as controlled by autonomous “agents” (drivers). These agents, then, are the players in the traffic game; the “strategy” in this case governs the choice of the path made by each agent; and the payoffs are taken as the negative travel cost (in a game, a players always try to maximize its payoff). In complex urban traffic situations, several kinds of games can be played. Players can be cooperative or non-cooperative. For instance, if two drivers compete for a place in the street to park their cars, they can try to see who will guide its car faster in a position to start maneuvering. They can initially not cooperate at all, but as soon as one perceives that the other has reached a good position, he would consider cooperating in order to avoid a crash. Sometimes players will cooperate with each other in order to achieve some common utility. A driver can be said to cooperate with others when he understands that placing his car in the middle of a crossroad can lead to an increase in traffic congestion, even if that action could have a slightly better payoff considering that he may have the chance to avoid one signal cycle (and thus reduce his overall traversing cost). The important notion here is that most of the time, drivers behave in such a way that they try to achieve a better payoff according to some utility. Here, we are mostly interested in describing traffic flows in a general fashion and thus, for our purposes, we will state

that drivers as self-interested agents are non-cooperative players in the sense that their utility is linked to trying to reach their destination as fast as they can, despite of other drivers’ travel costs and more importantly, despite of a good general functionality of the whole traffic system.

4.1 Model of the Transportation Network

To deal with the traffic problem in urban transportation network, we need a model of the infrastructure, which is the environment drivers interact in. First of all, we take a very simplistic view of the transportation network. Since we are interested in modeling urban traffic as flows, the network is then modeled as a directed graph G = (V, P) , where the vertex set V represents

intersections and the edge set P represents paths. Each path p ∈ P is governed by a latency

function lp(.) that describes the delay incurred by the traffic density on p. At this point, we get a question: what will then influence the time spent on a certain path? It seems natural that the time spent on each path depends on the traffic density on that path, which will mainly be a function of the number of cars, or traffic flow on the path, the traffic capacity of the path and the free flow travel time. The so called traffic capacity is an intrinsic property of an individual path. Since only a finite number of cars can travel along a path at any given time, the capacity of a path can then be defined as the maximum number of vehicles which can fit on it [1]. And the free flow travel time is the time spent to travel through the path when there are no other cars on that path [1]. Thus, given a path with a certain capacity, the time spent to traverse it increases with an increasing traffic flow on it. This infers that the path latency function lp(.) is a function of the traffic flow fp on that path. In this paper, we are not going to give a detailed mathematical definition of the path latency function. But, in general, the latency function of each path lp(fp) : P -> R+^ is traffic flow or congestion dependent. For the analysis that follows, we assume that lp(fp) is non-negative,

continuous, and non-decreasing, for each p ∈ P , which also says that the path latency will never

decrease when the traffic flow on it increases. Under the assumption that car drivers behave in a selfish manner, they will travel on the minimum-latency paths, given the perceived traffic congestion due to other road users. That is, the path latency function lp(fp) is the function that car drivers want to minimize for their own benefits.

Now, another question arises: what's the function the traffic authority wants to minimize in the transportation network? First of all, we introduce a notion: social cost, which measures the quality of the traffic assignment via the sum of the travel time in the transport network, or the total latency.

functions, the ratio of Nash cost to the optimal cost may not be bounded [2].

4.3 Congestion Pricing Mechanism

The inefficiency of selfish traffic assignment (and of Nash equilibriums more generally) motivates strategies for coping with selfishness—methods for ensuring that non-cooperative behavior results in a socially desirable outcome. Economists’ argue that this can be achieved with marginal cost pricing , which is now widely studied for influencing the selfish behavior in the internet. In Economics, marginal cost is defined as the additional cost of producing just one more ("marginal") unit of output:

Q

MC C

= ∆ ( ∆Q : the change in output, ∆C : the change in cost)

In the context of transportation network, for a path p with differentiable latency function lp , the marginal cost of increasing the flow of path p is defined as: lp * (fp) = ( fp · lp(fp) )’= lp(fp) + fp · lp’(fp), where lp * denotes the marginal cost function of path p. Then, the following lemma holds:

Lemma 4.2[2] Suppose P is a set of paths with differentiable latency functions l , and that fp · lp(fp)

is a convex function for each path. Then the traffic flow assignment f to P is socially optimal iff whenever i, j ∈P with f i > 0, li * (fi) ≤ lj * (fj).

In principle, the marginal cost pricing asserts that each driver should be charged for the additional delay ( fp · lp’(fp) )its presence causes for the other users on the path. Lemma 4.2 implies that it is possible to levy a toll on each path so that the resulting Nash equilibrium achieves the minimum- possible total latency under the assumption that all drivers choose paths to minimize the sum of the latency experienced and tolls paid ( lp(fp) + fp · lp’(fp) ). Thus, the definition of the payoff in the traffic game is changed from the negative travel time to the sum of the negative travel time and the negative toll paid.

As we know, in the absence of a control mechanism, individual drivers do not consider the congestion costs they impose on other drivers such that the whole traffic system functions in a sub optimal way. Inspired by the marginal cost pricing strategy, in the following we propose a congestion pricing mechanism for the traffic authority to eradicate such negative consequences of

selfish traffic assignment:

z The number of cars fp is monitored for each path; z The current latency and its gradient on each path is calculated to get the marginal cost ( lp(fp) + fp · lp’(fp) ).

z The congestion fee ( fp · lp’(fp) ) of each path is then broadcasted through the traffic information system, e.g. electronic information panels. z Charge drivers who actually enter the path p have to pay a congestion “fee” which is equal to fp · lp’(fp). Here, charging a congestion fee is not necessarily conducted in a monetary sense. It just represents an extra cost a driver has to pay for traveling along a particular path. For instance, it can be the time delay incurred by speed limit on that path.

Path Latency Marginal Cost 2 l 2 (f 2 )= 4 4 3 l 3 (f 3 )=f 3 2 x f 3

Figure 1: Road Structure Table 1: Latency and Marginal Cost Functions

In order to give a better illustration of the pricing mechanism discussed above, we build up a simple setting based on the road structure as shown in Figure 1: There are 5 paths numbered 1-5. At each entrance of the path, there is an electronic information panel giving information about the current traffic flow and the congestion fee for entering the path. Suppose, there are 4 drivers arriving at the east end of path 5. They can get to their destination via either path 2 or path 3. Table 1 gives their latency and marginal cost functions. In the absence of the congestion pricing mechanism, all drivers will choose to travel along path 3, because the latency is never longer than 4; thereby incurring a total latency of 16. However, with the pricing mechanism, although the first 2 drivers will still choose to travel along path 3, the rest 2 drivers will prefer path 2 because the

interactions between independent agents leading to the emergence of intelligent behavior, GT seems to be a good tool to represent situations that resemble a game between players trying to get the best payoff of their possible actions.

Curiosly, what we have just seen is that, in the present example of our application, local interactions can lead to global equilibria that are not necessarily desirable thus making it necessary to introduce an extra mechanism to regulate the behavior of agents; thanks to the conjuntion of the application of concepts from MAS and GT(especially the Nash equilibrium and Pareto efficiency) we could better understand a social phenomena and that was perhaps the greatest achievement of our efforts.

MAS can be applied to the study of social phenomena through computer simulations of these. In the present paper we took concepts from MAS and GT not to build up a simulation, but to enhance the understanding of a certain part of a social reality by using the notions and concepts that better characterize them. Inasmuch as a simplification of such phenomena was carried out, we think that the power of these approaches can be still experienced even if we are not dealing with the complex scenario that is Reality.

References:

1- Villiger, R. Game Theory and Traffic Control (Page 10-24)

2- Akella,A., Chawla, S. and Seshan,S. Mechanisms for Internet Routing: A Study

3- Burattim, E. and Cordeschi, R. Intelligenza Artificiale(in Italian) Chapter 5

4- Heath, D. C. Games and Social Contract Chapter 2