Download AI Notes Artificial Intelligence and more Lecture notes Artificial Intelligence in PDF only on Docsity!
Chapter 1
Overview of Artificial Intelligence
What is AI?
A rtificial Intelligence (AI) is a branch of Science which deals with helping machines find
solutions to complex problems in a more human-likefashion.
This generally involves borrowing characteristics from human intelligence, and applying
them as algorithms in a computer friendly way.
A more or less flexible or efficient approach can be taken depending on the requirements
established, which influences how artificial the intelligent behavior appears
Artificial intelligence can be viewed from a variety ofperspectives.
From the perspective of intelligence artificial
intelligence is making machines "intelligent" -- acting as we would expect people toact.
o The inability to distinguish computer responses from human responses is called the
Turingtest.
o Intelligence requires knowledge
o Expert problem solving - restricting domain to allow including significant
relevantknowledge
From a business perspective AI is a set of very powerful tools, and methodologies for using
those tools to solve businessproblems.
From a programming perspective, AI includes the study of symbolic programming,
problem solving, andsearch.
o Typically AI programs focus on symbols rather than numeric processing.
o Problem solving - achievegoals.
o Search - seldom access a solution directly. Search may include a variety oftechniques.
o AI programming languagesinclude:
- LISP, developed in the 1950s, is the early programming language
strongly associated with AI. LISP is a functional programming language with procedural
extensions. LISP (LISt Processor) was specifically designedfor
processing heterogeneous lists -- typically a list of symbols. Features of LISP are run- time
type checking, higher order functions (functions that have other functions as parameters),
automatic memory management (garbage collection) and an interactive environment.
- The second language strongly associated with AI is PROLOG.
PROLOG was developed in the 1970s. PROLOG is based on first order logic. PROLOG is
declarative in nature and has facilities for explicitly limiting the searchspace.
- Object-oriented languages are a class of languages more recently used
for AI programming. Important features of object-oriented languages include: concepts of
objects and messages, objects bundle data and methods for manipulating the data, sender
specifies what is to be done receiver decides how to do it, inheritance (object hierarchy
where objects inherit the attributes of the more general class of objects). Examples of object-
oriented languages are Smalltalk, Objective C, C++. Object oriented extensions to LISP
(CLOS - Common LISP Object System) and PROLOG (L&O - Logic & Objects) are
alsoused.
Artificial Intelligence is a new electronic machine that stores large amount of information
and process it at very highspeed
The computer is interrogated by a human via a teletype It passes if the human cannot tell if
there is a computer or human at the otherend
The ability to solveproblems
It is the science and engineering of making intelligent machines, especially intelligent
computer programs. It is related to the similar task of using computers to understand
humanintelligence
Importance of AI
Game Playing
You can buy machines that can play master level chess for a few hundred dollars. There is
some AI in them, but they play well against people mainly through brute force computation--
looking at hundreds of thousands of positions. To beat a world champion by brute force and
known reliable heuristics requires being able to look at 200 million positions persecond.
SpeechRecognition
Heuristic Classification
One of the most feasible kinds of expert system given the present knowledge of AI is to put
some information in one of a fixed set of categories using several sources of information. An
example is advising whether to accept a proposed credit card purchase. Information is
available about the owner of the credit card, his record of payment and also about the item he
is buying and about the establishment from which he is buying it (e.g., about whether there
have been previous credit card frauds at this establishment).
The applications of AI are shown in Fig1.1:
ConsumerMarketing
o Have you ever used any kind of credit/ATM/store card whileshopping?
o if so, you have very likely been “input” to an AIalgorithm
o All of this information is recordeddigitally
o Companies like Nielsen gather this information weekly and search for patterns
- general changes in consumerbehavior
- tracking responses to newproducts
- identifying customer segments: targeted marketing, e.g., they find out that consumers with
sports cars who buy textbooks respond well to offers of new credit cards.
o Algorithms (“data mining”) search data for patterns based on mathematical theories
oflearning
IdentificationTechnologies
o ID cards e.g., ATMcards
o can be a nuisance and security risk: cards can be lost, stolen, passwords forgotten,etc
o Biometric Identification, walk up to a lockeddoor
- Camera
- Fingerprintdevice
- Microphone
- Computer uses biometric signature foridentification
- Face, eyes, fingerprints, voicepattern
- This works by comparing data from person at door with stored library
- Learning algorithms can learn the matching process by analyzing a large library database
off-line, can improve itsperformance.
IntrusionDetection
o Computer security - we each have specific patterns of computer use times of day, lengths of
sessions, command used, sequence of commands,etc
- would like to learn the “signature” of each authorizeduser
- can identify non-authorizedusers
o How can the program automatically identifyusers?
- record user’s commands and timeintervals
- characterize the patterns for eachuser
- model the variability in thesepatterns
- classify (online) any new user by similarity to storedpatterns
Machine Translation
o Language problems in internationalbusiness
- e.g., at a meeting of Japanese, Korean, Vietnamese and Swedish investors, no
commonlanguage
- If you are shipping your software manuals to 127 countries, the solution is ; hire translators
totranslate
- would be much cheaper if a machine could dothis!
o How hard is automatedtranslation
- verydifficult!
- e.g., English toRussian
- not only must the words be translated, but their meaningalso!
Psychology/Cognitive Science
Biology/Neuroscience
Evolution
There is inevitably much overlap, e.g. between philosophy and logic, or between
mathematics and computation. By looking at each of these in turn, we can gain a better
understanding of their role in AI, and how these underlying disciplines have developed to
play thatrole.
Philosophy
~400 BC Socrates asks for an algorithm to distinguish piety fromnon-piety.
~350 BC Aristotle formulated different styles of deductive reasoning, which could
mechanically generate conclusions from initial premises, e.g. ModusPonens
If A?B and A then B
If A impliesB an
d
A is true then B is true when it’s raining you
get wet and it’s raining then you get wet
1596 – 1650 Rene Descartes idea of mind-body dualism – part of the mind is exempt from
physicallaws.
1646 – 1716 Wilhelm Leibnitz was one of the first to take the materialist position which
holds that the mind operates by ordinary physical processes – this has the implication that
mental processes can potentially be carried out bymachines.
Logic/Mathematics
Earl Stanhope’s Logic Demonstrator was a machine that was able to solve syllogisms,
numerical problems in a logical form, and elementary questions of probability.
1815 – 1864 George Boole introduced his formal language for making logical inference in
1847 – Booleanalgebra.
1848 – 1925 GottlobFrege produced a logic that is essentially the first-order logic that today
forms the most basic knowledge representationsystem.
1906 – 1978 Kurt Gödel showed in 1931 that there are limits to what logic can do. His
Incompleteness Theorem showed that in any formal logic powerful enough to describe the
properties of natural numbers, there are true statements whose truth cannot be established by
anyalgorithm.
1995 Roger Penrose tries to prove the human mind has non-computable capabilities.
Computation
1869 William Jevon’s Logic Machine could handle Boolean Algebra and Venn Diagrams,
and was able to solve logical problems faster than humanbeings.
1912 – 1954 Alan Turing tried to characterise exactly which functions are capable of being
computed. Unfortunately it is difficult to give the notion of computation a formal definition.
However, the Church-Turing thesis, which states that a Turing machine is capable of
computing any computable function, is generally accepted as providing a sufficient
definition. Turing also showed that there were some functions which no Turing machine can
compute (e.g. Halting Problem).
1903 – 1957 John von Neumann proposed the von Neuman architecture which allows a
description of computation that is independent of the particular realisation of the computer.
1960s Two important concepts emerged: Intractability (when solution time grows at least
exponentially) and Reduction (to ‘easier’problems).
Psychology / CognitiveScience
Modern Psychology / Cognitive Psychology / Cognitive Science isthe science which studies
how the mind operates, how we behave, and how our brains process information.
Language is an important part of human intelligence. Much of the early work on knowledge
representation was tied to language and informed by research into linguistics.
It is natural for us to try to use our understanding of how human (and other animal) brains
lead to intelligent behavior in our quest to build artificial intelligent systems. Conversely, it
makes sense to explore the properties of artificial systems (computer models/simulations) to
test our hypotheses concerning humansystems.
Many sub-fields of AI are simultaneously building models of how the human system
operates, and artificial systems for solving real world problems, and are allowing useful
ideas to transfer betweenthem.
Biology /Neuroscience
Neural Networks – e.g. brain modelling, time series prediction,classification
Evolutionary Computation – e.g. genetic algorithms, geneticprogramming
Vision – e.g. object recognition, imageunderstanding
Robotics – e.g. intelligent control, autonomousexploration
Expert Systems – e.g. decision support systems, teachingsystems
Speech Processing– e.g. speech recognition andproduction
Natural Language Processing – e.g. machinetranslation
Planning – e.g. scheduling, gameplaying
Machine Learning – e.g. decision tree learning, version spacelearning
SpeechProcessing
As well as trying to understand human systems, there are also numerous real world
applications: speech recognition for dictation systems and voice activated control; speech
production for automated announcements and computerinterfaces.
How do we get from sound waves to text streams andvice-versa?
Natural LanguageProcessing
For example, machine understanding and translation of simplesentences:
Planning
Planning refers to the process of choosing/computing the correct sequence ofsteps to solve a
given problem.
To do this we need some convenient representation of the problem domain. We can define
states in some formal language, such as a subset of predicate logic, or a series ofrules.
A plan can then be seen as a sequence of operations that transform the initial state into the
goal state, i.e. the problem solution. Typically we will use some kind of search algorithm to
find a goodplan.
CommonTechniques
Even apparently radically different AI systems (such as rule based expert systems and neural
networks) have many commontechniques.
Four important onesare:
o Knowledge Representation: Knowledge needs to berepresented somehow – perhaps as a
series of if-then rules, as a frame based system, as a semantic network, or in the connection
weights of an artificial neural network.
o Learning: Automatically building up knowledge from the environment – such as acquiring
the rules for a rule based expert system, or determining the appropriate connection weights in
an artificial neuralnetwork.
o Rule Systems: These could be explicitly built into an expert system by a knowledge engineer,
or implicit in the connection weights learnt by a neuralnetwork.
o Search: This can take many forms – perhaps searching for a sequence of states that leads
quickly to a problem solution, or searching for a good set of connection weights for a neural
network by minimizing a fitness function.
AI and related fields
LogicalAI
What a program knows about the world in general the facts of the specific situation in which
it must act, and its goals are all represented by sentences of some mathematical logical
language. The program decides what to do by inferring that certain actions are appropriate
for achieving its goals.
Search
AI programs often examine large numbers of possibilities, e.g. moves in a chess game or
inferences by a theorem proving program. Discoveries are continually made about how to do
this more efficiently in various domains.
PatternRecognition
When a program makes observations of some kind, it is often programmed to compare what
it sees with a pattern. For example, a vision program may try to match a pattern ofeyes and a
nose in a scene in order to find a face. More complex patterns,
e.g. in a natural language text, in a chess position, or in the history of some event are also
studied.
This is a study of the kinds of knowledge that are required for solving problems in the world.
Ontology
Ontology is the study of the kinds of things that exist. In AI, the programs and sentences deal
with various kinds of objects, and we study what these kinds are and what their basic
properties are. Emphasis on ontology begins in the 1990s.
Heuristics
A heuristic is a way of trying to discover something or an idea imbedded in a program. The
term is used variously in AI. Heuristic functions are used in some approaches to search to
measure how far a node in a search tree seems to be from a goal. Heuristic predicates that
compare two nodes in a search tree to see if one is better than the other, i.e. constitutes an
advance toward the goal, may be moreuseful.
Genetic Programming
Genetic programming is a technique for getting programs to solve a task by mating random
Lisp programs and selecting fittest in millions of generations.
Chapter 2
Search and Control Strategies:
Problem solving is an important aspect of Artificial Intelligence. A problem can be
considered to consist of a goal and a set of actions that can be taken to lead to the goal. Atany
given time, we consider the state of the search space to represent where we have reached as a
result of the actions we have applied so far. For example, consider the problem of looking
for a contact lens on a football field. The initial state is how we start out, which is to say we
know that the lens is somewhere on the field, but we don’t know where. If we use the
representation where we examine the field in units of one square foot, then our first action
might be to examine the square in the top-left corner of the field. If we do not find the lens
there, we could consider the state now to be that we have examined the top-left square and
have not found the lens. After a number of actions, the state might be that we have examined
500 squares, and we have now just found the lens in the last square we examined. This is a
goal state because it satisfies the goal that we had of finding a contactlens.
Search is a method that can be used by computers to examine a problem space
likethis in order to find a goal. Often, we want to find the goal as quickly as possible or
without usingtoomanyresources.Aproblemspacecanalsobeconsideredtobeasearchspace
o Indexing schemes (e.g.,B-trees)
State Space Representations: The state space is simply the space of all possible states, or
configurations, that our system may be in. Generally, of course, we prefer to work with some
convenient representation of that searchspace.
There are two components to the representation of statespaces:
Static States
Transitions betweenStates
State Space Graphs: If the number of possible states of the system is small enough, we can
represent all of them, along with the transitions between them, in a state space graph,e.g.
Routes through State Space: Our general aim is to search for a route, or sequence of
transitions, through the state space graph from our initial state to a goalstate.
Sometimes there will be more than one possible goal state. We define a goal test to
determine if a goal state has beenachieved.
The solution can be represented as a sequence of link labels (or transitions) on the state space
graph. Note that the labels depend on the direction moved along thelink.
Sometimes there may be more than one path to a goal state, and we may want to find the
optimal (best possible) path. We can define link costs and path costs for measuring the cost
of going along a particular path, e.g. the path cost may just equal the number of links, or
could be the sum of individual linkcosts.
For most realistic problems, the state space graph will be too large for us to hold all of it
explicitly in memory at any onetime.
Search Trees: It is helpful to think of the search process as building up a search tree of
routes through the state space graph. The root of the search tree is the search node
corresponding to the initialstate.
The leaf nodes correspond either to states that have not yet been expanded, or to states that
generated no further nodes whenexpanded.
At each step, the search algorithm chooses a new unexpanded leaf node to expand. The
different search strategies essentially correspond to the different algorithms one can use to
select which is the next mode to be expanded at eachstage.
Examples of search problems
Traveling Salesman Problem: G
G
i
i v
v e
e n
n n
n c
c i
i t
t i
i e
e s
s w
w i
i t
t h
h k
k n
n o
o w
w n
n d
d i
i s
s t
t a
a n
n c
c e
e s
s b
b e
e t
t w
w e
e e
e n
n e
e a
a c
c h
h
p paaiirr,,ffiinnddtthheesshhoorrtteessttttoouurrtthhaattppaasssseesstthhrroouugghhaalllltthheecciittiieesseexxaaccttllyyoonncceebbeeffoorree
r
r e
e t
t u
u r
r n
n i
i n
n g
g t
t o
o t
t h
h e
e s
s t
t a
a r
r t
t i
i n
n g
g c
c i
i t
t y
y .
A Alloowweerrbboouunnddoonntthheelleennggtthhllooffaannyyttoouurrccaannbbeeccoommppuutteeddaassffoolllloowwss
FFoorreeaacchhcciittyyii,, 11 ≤ i ≤ n, find the sum s i
of the distances from city ito the two nearestcities.
Compute the sum s of these nnumbers.
Divide the result by 2 and round up the result to the nearestinteger lb= s /
The lower bound for the graph shown in the Fig 5.1 can be computed asfollows:
Root node includes only the starting vertex a with a lower bound of lb = [(1 + 3)
+ (3 + 6) + (1 + 2) + (3 + 4) + (2 + 3)] / 2 = 14.
Node 1 represents the inclusion of edge (a,b)
lb = [(1 + 3) + (3 + 6) + (1 + 2) + (3 + 4) + (2 + 3)] / 2 = 14.
Node 2 represents the inclusion of edge (a, c). Since b is not visited before c, this node
isterminated.
Node 3 represents the inclusion of edge (a,d)
lb = [(1 + 5) + (3 + 6) + (1 + 2) + (3 + 5) + (2 + 3)] / 2 = 16.
Node 1 represents the inclusion of edge (a,e)
lb = [(1 + 8) + (3 + 6) + (1 + 2) + (3 + 4) + (2 + 8)] / 2 = 19.
Among all the four live nodes of the root, node 1 has a better lower bound. Hence we branch
from node1.
Node 5 represents the inclusion of edge (b,c)
lb = [(1 + 3) + (3 + 6) + (1 + 6) + (3 + 4) + (2 + 3)] / 2 = 16.
Node 6 represents the inclusion of edge (b,d)
lb = [(1 + 3) + (3 + 7) + (1 + 2) + (3 + 7) + (2 + 3)] / 2 = 16.
Node 7 represents the inclusion of edge (b,e)
lb = [(1 + 3) + (3 + 9) + (1 + 2) + (3 + 4) + (2 + 9)] / 2 = 19.
Since nodes 5 and 6 both have the same lower bound, we branch out from each ofthem.
Node 8 represents the inclusion of the edges (c, d), (d, e) and (e, a). Hence, the length of
thetour,
l = 3 + 6 + 4 + 3 + 8 = 24.
Node 9 represents the inclusion of the edges (c, e), (e, d) and (d, a). Hence, the length of
thetour,
l = 3 + 6 + 2 + 3 + 5 = 19.
Node 10 represents the inclusion of the edges (d, c), (c, e) and (e, a). Hence, the length of
thetour,
l = 3 + 7 + 4 + 2 + 8 = 24.
Node 11 represents the inclusion of the edges (d, e), (e, c) and (c, a). Hence, the length of
thetour,
l = 3 + 7 + 3 + 2 + 1 = 16.
Node 11 represents an optimal tour since its tour length is better than or equal to the other
live nodes, 8, 9, 10, 3 and4.
The optimal tour is a → b → d → e → c → a with a tour length of16.
Uniformed or Blind search
Breadth First Search (BFS): BFS expands the leaf node with the lowest path cost so far,
and keeps going until a goal node is generated. If the path cost simply equals the number of
links, we can implement this as a simple queue (“first in, firstout”).
