Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
AI NOTES FOR COMPUTER SCIENCE ENGINEERING
Typology: Thesis
1 / 213
This page cannot be seen from the preview
Don't miss anything!
(Subject Code: BCS-404)
DISCLAIMER
This document does not claim any originality and cannot be used as a substitute for prescribed textbooks. The information presented here is merely a collection by the committee members for their respective teaching assignments. Various sources as mentioned at the end of the document as well as freely available material from internet were consulted for preparing this document. The ownership of the information lies with the respective authors or institutions.
A rtificial Intelligence (AI) is a branch of Science which deals with helping machines find solutions to complex problems in a more human-like fashion. This generally involves borrowing characteristics from human intelligence, and applying them as algorithms in a computer friendly way. A more or less flexible or efficient approach can be taken depending on the requirements established, which influences how artificial the intelligent behavior appears Artificial intelligence can be viewed from a variety of perspectives. From the perspective of intelligence artificial intelligence is making machines "intelligent" -- acting as we would expect people to act. o The inability to distinguish computer responses from human responses is called the Turing test. o Intelligence requires knowledge o Expert problem solving - restricting domain to allow including significant relevant knowledge From a business perspective AI is a set of very powerful tools, and methodologies for using those tools to solve business problems. From a programming perspective, AI includes the study of symbolic programming, problem solving, and search. o Typically AI programs focus on symbols rather than numeric processing. o Problem solving - achieve goals. o Search - seldom access a solution directly. Search may include a variety of techniques. o AI programming languages include:
processing heterogeneous lists -- typically a list of symbols. Features of LISP are run- time type checking, higher order functions (functions that have other functions as parameters), automatic memory management (garbage collection) and an interactive environment.
Game Playing You can buy machines that can play master level chess for a few hundred dollars. There is some AI in them, but they play well against people mainly through brute force computation--looking at hundreds of thousands of positions. To beat a world champion by brute force and known reliable heuristics requires being able to look at 200 million positions per second.
Speech Recognition
Heuristic Classification One of the most feasible kinds of expert system given the present knowledge of AI is to put some information in one of a fixed set of categories using several sources of information. An example is advising whether to accept a proposed credit card purchase. Information is available about the owner of the credit card, his record of payment and also about the item he is buying and about the establishment from which he is buying it (e.g., about whether there have been previous credit card frauds at this establishment).
The applications of AI are shown in Fig 1.1: Consumer Marketing o Have you ever used any kind of credit/ATM/store card while shopping? o if so, you have very likely been “input” to an AI algorithm o All of this information is recorded digitally o Companies like Nielsen gather this information weekly and search for patterns
Psychology/Cognitive Science Biology/Neuroscience Evolution There is inevitably much overlap, e.g. between philosophy and logic, or between mathematics and computation. By looking at each of these in turn, we can gain a better understanding of their role in AI, and how these underlying disciplines have developed to play that role. Philosophy ~400 BC Socrates asks for an algorithm to distinguish piety from non-piety. ~350 BC Aristotle formulated different styles of deductive reasoning, which could mechanically generate conclusions from initial premises, e.g. Modus Ponens If A? B and A then B If A implies B and A is true then B is true when it’s raining you get wet and it’s raining then you get wet 1596 – 1650 Rene Descartes idea of mind-body dualism – part of the mind is exempt from physical laws. 1646 – 1716 Wilhelm Leibnitz was one of the first to take the materialist position which holds that the mind operates by ordinary physical processes – this has the implication that mental processes can potentially be carried out by machines. Logic/Mathematics Earl Stanhope’s Logic Demonstrator was a machine that was able to solve syllogisms, numerical problems in a logical form, and elementary questions of probability. 1815 – 1864 George Boole introduced his formal language for making logical inference in 1847 – Boolean algebra. 1848 – 1925 Gottlob Frege produced a logic that is essentially the first-order logic that today forms the most basic knowledge representation system. 1906 – 1978 Kurt Gödel showed in 1931 that there are limits to what logic can do. His Incompleteness Theorem showed that in any formal logic powerful enough to describe the properties of natural numbers, there are true statements whose truth cannot be established by any algorithm.
1995 Roger Penrose tries to prove the human mind has non-computable capabilities. Computation 1869 William Jevon’s Logic Machine could handle Boolean Algebra and Venn Diagrams, and was able to solve logical problems faster than human beings. 1912 – 1954 Alan Turing tried to characterise exactly which functions are capable of being computed. Unfortunately it is difficult to give the notion of computation a formal definition. However, the Church-Turing thesis, which states that a Turing machine is capable of computing any computable function, is generally accepted as providing a sufficient definition. Turing also showed that there were some functions which no Turing machine can compute (e.g. Halting Problem). 1903 – 1957 John von Neumann proposed the von Neuman architecture which allows a description of computation that is independent of the particular realisation of the computer. 1960s Two important concepts emerged: Intractability (when solution time grows atleast exponentially) and Reduction (to ‘easier’ problems). Psychology / Cognitive Science Modern Psychology / Cognitive Psychology / Cognitive Science is the science which studies how the mind operates, how we behave, and how our brains process information. Language is an important part of human intelligence. Much of the early work on knowledge representation was tied to language and informed by research into linguistics. It is natural for us to try to use our understanding of how human (and other animal) brains lead to intelligent behavior in our quest to build artificial intelligent systems. Conversely, it makes sense to explore the properties of artificial systems (computer models/simulations) to test our hypotheses concerning human systems. Many sub-fields of AI are simultaneously building models of how the human system operates, and artificial systems for solving real world problems, and are allowing useful ideas to transfer between them. Biology / Neuroscience
Neural Networks – e.g. brain modelling, time series prediction, classification Evolutionary Computation – e.g. genetic algorithms, genetic programming Vision – e.g. object recognition, image understanding Robotics – e.g. intelligent control, autonomous exploration Expert Systems – e.g. decision support systems, teaching systems Speech Processing– e.g. speech recognition and production Natural Language Processing – e.g. machine translation Planning – e.g. scheduling, game playing Machine Learning – e.g. decision tree learning, version space learning Speech Processing As well as trying to understand human systems, there are also numerous real world applications: speech recognition for dictation systems and voice activated control; speech production for automated announcements and computer interfaces. How do we get from sound waves to text streams and vice-versa?
Natural Language Processing For example, machine understanding and translation of simple sentences: Planning Planning refers to the process of choosing/computing the correct sequence of steps to solve a given problem. To do this we need some convenient representation of the problem domain. We can define states in some formal language, such as a subset of predicate logic, or a series of rules. A plan can then be seen as a sequence of operations that transform the initial state into the goal state, i.e. the problem solution. Typically we will use some kind of search algorithm to find a good plan. Common Techniques Even apparently radically different AI systems (such as rule based expert systems and neural networks) have many common techniques. Four important ones are:
o Knowledge Representation: Knowledge needs to be represented somehow – perhaps as a series of if-then rules, as a frame based system, as a semantic network, or in the connection weights of an artificial neural network. o Learning: Automatically building up knowledge from the environment – such as acquiring the rules for a rule based expert system, or determining the appropriate connection weights in an artificial neural network. o Rule Systems: These could be explicitly built into an expert system by a knowledge engineer, or implicit in the connection weights learnt by a neural network. o Search: This can take many forms – perhaps searching for a sequence of states that leads quickly to a problem solution, or searching for a good set of connection weights for a neural network by minimizing a fitness function.
Logical AI What a program knows about the world in general the facts of the specific situation in which it must act, and its goals are all represented by sentences of some mathematical logical language. The program decides what to do by inferring that certain actions are appropriate for achieving its goals.
Search AI programs often examine large numbers of possibilities, e.g. moves in a chess game or inferences by a theorem proving program. Discoveries are continually made about how to do this more efficiently in various domains.
Pattern Recognition When a program makes observations of some kind, it is often programmed to compare what it sees with a pattern. For example, a vision program may try to match a pattern of eyes and a nose in a scene in order to find a face. More complex patterns, e.g. in a natural language text, in a chess position, or in the history of some event are also studied.
This is a study of the kinds of knowledge that are required for solving problems in the world.
Ontology Ontology is the study of the kinds of things that exist. In AI, the programs and sentences deal with various kinds of objects, and we study what these kinds are and what their basic properties are. Emphasis on ontology begins in the 1990s.
Heuristics A heuristic is a way of trying to discover something or an idea imbedded in a program. The term is used variously in AI. Heuristic functions are used in some approaches to search to measure how far a node in a search tree seems to be from a goal. Heuristic predicates that compare two nodes in a search tree to see if one is better than the other, i.e. constitutes an advance toward the goal, may be more useful.
Genetic Programming Genetic programming is a technique for getting programs to solve a task by mating random Lisp programs and selecting fittest in millions of generations. Search and Control Strategies:
Problem solving is an important aspect of Artificial Intelligence. A problem can be considered to consist of a goal and a set of actions that can be taken to lead to the goal. At any given time, we consider the state of the search space to represent where we have reached as a result of the actions we have applied so far. For example, consider the problem of looking for a contact lens on a football field. The initial state is how we start out, which is to say we know that the lens is somewhere on the field, but we don’t know where. If we use the representation where we examine the field in units of one square foot, then our first action might be to examine the square in the top-left corner of the field. If we do not find the lens there, we could consider the state now to be that we have examined the top-left square and have not found the lens. After a number of actions, the state might be that we have examined 500 squares, and we have now just found the lens in the last square we examined. This is a goal state because it satisfies the goal that we had of finding a contact lens.
Search is a method that can be used by computers to examine a problem space like this in order to find a goal. Often, we want to find the goal as quickly as possible or without using too many resources. A problem space can also be considered to be a search space
because in order to solve the problem, we will search the space for a goal state.We will continue to use the term search space to describe this concept. In this chapter, we will look at a number of methods for examining a search space. These methods are called search methods.
The Importance of Search in AI It has already become clear that many of the tasks underlying AI can be phrased in terms of a search for the solution to the problem at hand. Many goal based agents are essentially problem solving agents which must decide what to do by searching for a sequence of actions that lead to their solutions. For production systems, we have seen the need to search for a sequence of rule applications that lead to the required fact or action. For neural network systems, we need to search for the set of connection weights that will result in the required input to output mapping. Which search algorithm one should use will generally depend on the problem domain? There are four important factors to consider: Completeness – Is a solution guaranteed to be found if at least one solution exists? Optimality – Is the solution found guaranteed to be the best (or lowest cost) solution if there exists more than one solution? Time Complexity – The upper bound on the time required to find a solution, as a function of the complexity of the problem. Space Complexity – The upper bound on the storage space (memory) required at any point during the search, as a function of the complexity of the problem. Preliminary concepts
Two varieties of space-for-time algorithms: Input enhancement — preprocess the input (or its part) to store some info to be used later in solving the problem o Counting for sorting o String searching algorithms Prestructuring — preprocess the input to make accessing its elements easier o Hashing
For most realistic problems, the state space graph will be too large for us to hold all of it explicitly in memory at any one time. Search Trees: It is helpful to think of the search process as building up a search tree of routes through the state space graph. The root of the search tree is the search node corresponding to the initial state. The leaf nodes correspond either to states that have not yet been expanded, or to states that generated no further nodes when expanded.
At each step, the search algorithm chooses a new unexpanded leaf node to expand. The different search strategies essentially correspond to the different algorithms one can use to select which is the next mode to be expanded at each stage.
Examples of search problems
Traveling Salesman Problem: GGiivveenn nn cciittiieess wwiitthh kknnoowwnn ddiissttaanncceess bbeettwweeeenn eeaacchh ppaaiirr,, ffiinndd tthhee sshhoorrtteesstt ttoouurr tthhaatt ppaasssseess tthhrroouugghh aallll tthhee cciittiieess eexxaaccttllyy oonnccee bbeeffoorree rreettuurrnniinngg ttoo tthhee ssttaarrttiinngg cciittyy.. (^) AA lloowweerr bboouunndd oonn tthhee lleennggtthh ll ooff aannyy ttoouurr ccaann bbee ccoommppuutteedd aass ffoolllloowwss (^) FFoorr eeaacchh cciittyy ii,, 1 1 ≤ i ≤ n, find the sum si of the distances from city i to the two nearest cities. Compute the sum s of these n numbers. Divide the result by 2 and round up the result to the nearest integer lb = s / 2
The lower bound for the graph shown in the Fig 5.1 can be computed as follows:
lb = [(1 + 3) + (3 + 6) + (1 + 2)
For any subset of tours that must include particular edges of a given graph, the lower bound can be modified accordingly. E.g.: For all the Hamiltonian circuits of the graph that must include edge (a, d), the lower bound can be computed as follows: lb = [(1 + 5) + (3 + 6) + (1 + 2) + (3 + 5) + (2 + 3)] / 2 = 16.
Applying the branch-and-bound algorithm, with the bounding function lb = s / 2, to find the shortest Hamiltonian circuit for the given graph, we obtain the state-space tree as shown below: To reduce the amount of potential work, we take advantage of the following two observations: We can consider only tours that start with a. Since the graph is undirected, we can generate only tours in which b is visited before c. In addition, after visiting n – 1 cities, a tour has no choice but to visit the remaining unvisited city and return to the starting one is shown in the Fig 5.
