Applications - Artificial Intelligence - Lecture Slides, Slides of Artificial Intelligence

Download Applications - Artificial Intelligence - Lecture Slides and more Slides Artificial Intelligence in PDF only on Docsity!

Lecture 7 of 41

AI Applications 1 of 3

and Heuristic Search by Iterative Improvement

Lecture Outline

• Today’s Reading
• Reading for Next Week: Chapter 6, Russell and Norvig
• More Heuristic Search
  • Best-First Search: A/A concluded*
  • Iterative improvement
    • Hill-climbing
    • Simulated annealing (SA)
  • Search as function maximization
    • Problems: ridge; foothill; plateau, jump discontinuity
    • Solutions: macro operators; global optimization (genetic algorithms / SA)
• Next Lecture: Constraint Satisfaction Search, Heuristics (Concluded)
• Next Week: Adversarial Search (e.g., Game Tree Search)
  • Competitive problems
  • Minimax algorithm

AI Applications [2]:

Games

• Human-Level AI in Games

• Turn-Based: Empire, Strategic Conquest, Civilization, Empire Earth

• “Real-Time” Strategy: Warcraft/Starcraft, TA, C&C, B&W, etc.

• Combat Games and Simulation

• DOOM 3, Quake 3, HALO 2, Unreal Tournament (UT) 2004, etc.
• Side-scrollers

• Role-Playing Games (RPGs)

• Roguelikes: Rogue; NetHack, Slash’Em; Moria, Angband; ADOM
• Small-scale multiplayer RPGs: Diablo
• MUDs, MUSHes, MOOs
• Massively-Multiplayer Online RPGs (MMORPGs)

• Other Board Games, Strategy Games

• Abstract, Mathematical, Logical Games: Golub, Smullyan, etc.

AI Applications [3]:

Training and Tutoring

Virtual Environments for

Immersive Training

Normal

Ignited

Engulfed

Destroyed

Extinguished

Fire Alarm

Flooding

DC-ARM – http://www-kbs.ai.uiuc.edu © 1997 University of Illinois

Search-Based Problem Solving:

Quick Review

• function General-Search ( problem, strategy ) returns a solution or failure
  • Queue: represents search frontier (see: Nilsson – OPEN / CLOSED lists)
  • Variants: based on “add resulting nodes to search tree”
• Previous Topics
  • Formulating problem
  • Uninformed search
    • No heuristics: only g ( n ), if any cost function used
    • Variants: BFS (uniform-cost, bidirectional), DFS (depth-limited, ID-DFS)
  • Heuristic search
    • Based on h – (heuristic) function, returns estimate of min cost to goal
    • h only: greedy ( aka myopic) informed search
    • _A/A: f(n) = g(n) + h(n)_* – frontier based on estimated + accumulated cost
• Today: More Heuristic Search Algorithms
  • A extensions: iterative deepening (IDA) and simplified memory-bounded (SMA)*
  • Iterative improvement: hill-climbing, MCMC (simulated annealing)
  • Problems and solutions (macros and global optimization)

Properties of Algorithm A/A*:

Review

• Admissibility: Requirement for A Search to Find Min-Cost Solution*
• Related Property: Monotone Restriction on Heuristics
  • For all nodes m , n such that m is a descendant of n : h(m)  h(n) - c(n, m)
  • Discussion questions
    • Admissibility  monotonicity? Monotonicity  admissibility?
    • What happens if monotone restriction is violated? (Can we fix it?)
• Optimality Proof for Admissible Heuristics
  • Theorem: If  n****. h ( n )  _h_* ( n ) _, A will never return a suboptimal goal node._*
  • Proof
    • Suppose _A_* returns x such that  s****. g ( s ) < g ( x )
    • Let path from root to s be < n 0 , n 1 , …, nk > where nk  s
    • Suppose _A_* expands a subpath < n 0 , n 1 , …, nj > of this path
    • Lemma: by induction on i , s = nk is expanded as well

Base case: n 0 (root) always expanded

Induction step: h ( nj +1)  _h_* ( nj +1), so f ( nj +1)  f ( x ), Q.E.D.

• Contradiction: if s were expanded, A would have selected* s , not x

Iterative Improvement:

Framework

• Intuitive Idea
  • “Single-point search frontier”
    • Expand one node at a time
    • Place children at head of queue
    • Sort only this sublist, by f
  • Result – direct convergence in direction of steepest:
    • Ascent (in criterion)
    • Descent (in error)
  • Common property: proceed toward goal from search locus (or loci)
• Variations
  • Local (steepest ascent hill-climbing) versus global (simulated annealing)
  • Deterministic versus Monte-Carlo
  • Single-point versus multi-point
    • Maintain frontier
    • Systematic search (cf. OPEN / CLOSED lists): parallel simulated annealing
    • Search with recombination: genetic algorithm

Hill-Climbing [1]:

An Iterative Improvement Algorithm

• function Hill-Climbing ( problem ) returns solution state
  • inputs: problem : specification of problem (structure or class)
  • static: current , next : search nodes
  • current  Make-Node ( problem****. Initial-State )
  • loop do
    • next  a highest-valued successor of current
    • if next.value () < current.value () then return current
    • current  next // make transition
  • end
• Steepest Ascent Hill-Climbing
  • aka gradient ascent (descent)
  • Analogy: finding “tangent plane to objective surface”
  • Implementations
    • Finding derivative of (differentiable) f with respect to parameters
    • Example: error backpropagation in artificial neural networks (later)
• Discussion: Difference Between Hill-Climbing, Best-First?

0 1 w n

E

w

E

w

E

E w 

Hill-Climbing [3]:

Local Optima (Foothill Problem)

• Local Optima aka Local Trap States
• Problem Definition
  • Point reached by hill-climbing may be maximal but not maximum
  • Maximal
    • Definition: not dominated by any neighboring point (with respect to criterion

measure)

• In this partial ordering, maxima are incomparable
• Maximum
• Definition: dominates all neighboring points ( wrt criterion measure)
• Different partial ordering imposed: “ z value”
• Ramifications
• Steepest ascent hill-climbing will become trapped ( why? )
• Need some way to break out of trap state
• Accept transition (i.e., search move) to dominated neighbor
• Start over: random restarts

Hill-Climbing [4]:

Lack of Gradient (Plateau Problem)

• Zero Gradient Neighborhoods aka Plateaux
• Problem Definition
  • Function space may contain points whose neighbors are indistinguishable ( wrt

criterion measure)

• Effect: “flat” search landscape
• Discussion
  • When does this happen in practice?
  • Specifically, for what kind of heuristics might this happen?
• Ramifications
• Steepest ascent hill-climbing will become trapped ( why? )
• Need some way to break out of zero gradient
• Accept transition (i.e., search move) to random neighbor
• Random restarts
• Take bigger steps (later, in planning)

Ridge Problem Solution:

Multi-Step Trajectories (Macros)

• Intuitive Idea: Take More than One Step in Moving along Ridge
• Analogy: Tacking in Sailing
  • Need to move against wind direction
  • Have to compose move from multiple small steps
    • Combined move : in (or more toward) direction of steepest gradient
    • Another view: decompose problem into self-contained subproblems
• Multi-Step Trajectories: Macro Operators
  • Macros: (inductively) generalize from 2 to > 2 steps
  • Example: Rubik’s Cube
    • Can solve 3 x 3 x 3 cube by solving, interchanging 2 x 2 x 2 cubies
    • Knowledge used to formulate subcube (cubie) as macro operator
  • Treat operator as single step (multiple primitive steps)
• Discussion: Issues
  • How can we be sure macro is atomic? What are pre-, postconditions?
  • What is good granularity (length in primitives) for macro in our problem?

Plateau, Local Optimum, Ridge Solution:

Global Optimization

• Intuitive Idea
  • Allow search algorithm to take some “bad” steps to escape from trap states
  • Decrease probability of taking such steps gradually to prevent return to traps
• Analogy: Marble(s) on Rubber Sheet
  • Goal: move marble(s) into global minimum from any starting position
  • Shake system: hard at first, gradually decreasing vibration
  • Marbles tend to break out of local minima but have less chance of re-entering
• Analogy: Annealing
  • Ideas from metallurgy, statistical thermodynamics
  • Cooling molten substance: slow as opposed to rapid (quenching)
  • Goal: maximize material strength of solidified substance (e.g., metal or glass)
• Multi-Step Trajectories in Global Optimization: Super-Transitions
• Discussion: Issues
  • How can we be sure macro is atomic? What are pre-, postconditions?
  • What is good granularity (length in primitives) for macro in our problem?

Iterative Improvement

Global Optimization (GO) Algorithms

• Idea: Apply Global Optimization with Iterative Improvement
  • Iterative improvement: local transition (primitive step)
  • Global optimization algorithm
    • “Schedules” exploration of landscape
    • Selects next state to visit
    • Guides search by specifying probability distribution over local transitions
• Brief History of Markov Chain Monte Carlo (MCMC) Family
  • MCMC algorithms first developed in 1940s (Metropolis)
  • First implemented in 1980s
    • “Optimization by simulated annealing” (Kirkpatrick, Gelatt, Vecchi, 1983)
    • Boltzmann machines (Ackley, Hinton, Sejnowski, 1985)
  • Tremendous amount of research and application since
    • Neural, genetic, Bayesian computation
    • See: CIS730 Class Resources page

Simulated Annealing [1]:

An Iterative Improvement GO Algorithm

• function Simulated-Annealing ( problem, schedule ) returns solution state
  • inputs: problem : specification of problem (structure or class)

schedule : mapping from time to “temperature” (scalar)

• static: current , next : search nodes

T : “temperature” controlling probability of downward steps

• current  Make-Node ( problem****. Initial-State )
• for i  1 to  do
  • T  schedule [ t ]
  • if T = 0 then return current
  • next  a randomly selected successor of current
  •  E  next.value () – current.value ()
  • if  E > 0 then current  next
  • else current  next only with probability e  E/T
• General Theoretical Properties of Simulated Annealing (and Other MCMC)
• Converges in probability to global optimum (time: domain-dependent)
• Not guaranteed to find solution in given number of steps