OHJ-2556 ARTIFICIAL INTELLIGENCE Spring 2011 1 2 General 6 credit units Can be included in post-graduate studies Lectures (4h per week), 7 + 6½ weeks Student presentations instead of lectures towards the end Weekly exercises (2h per week) Programming exercise An essay You may have a chance to influence the contents now Lectures based on a new edition of the course book, not significantly different from the previous one 1
3 Organization & timetable Lectures: prof. Tapio Elomaa Tue 12 14 TB219 & Thu 12 14 TB223 Jan. 11 Apr. 19 Period break: Feb. 28 Mar. 6 Easter break: Apr. 21 27 Student presentations after the Easter break Weekly exercises starting on week 4 (Jan. 25 ) M.Sc. Teemu Heinimäki Tue 16 18 TC163 Programming exercises: M.Sc. Timo Aho Exam: Wed May 11, 2011 9 12 AM 4 Topics Part I Artificial Intelligence 1 Introduction 2 Intelligent Agents Part II Problem Solving 3 Solving Problems by Searching 4 Beyond Classical Search 5 Adversarial Search 6 Constraint Satisfaction Problems Part III Knowledge and Reasoning 7 Logical Agents 8 First-Order Logic 9 Inference in First-Order Logic 10 Classical Planning 11 Planning and Acting in the Real World 12 Knowledge Representation Part IV Uncertain Knowledge and Reasoning 13 Quantifying Uncertainty 14 Probabilistic Reasoning 15 Probabilistic Reasoning over Time 16 Making Simple Decisions 17 Making Complex Decisions Part V Learning 18 Learning from Examples 19 Knowledge in Learning 20 Learning Probabilistic Models 21 Reinforcement Learning Part VII Communicating, Perceiving, and Acting 22 Natural Language Processing 23 Natural Language for Communication 24 Perception 25 Robotics Part VIII Conclusions 26 Philosophical Foundations 27 AI: The Present and Future 2
5 Exercises, details still open The course has two compulsory exercises. To pass the course you need to pass both of them Both exercises are graded on scale 0 10, in addition the presentation yields up to 4 points Grade 1 means passing 1. Programming exercise?. Group/single? Returned by date? 2. Essay. Topic of your own choice from the area of AI. Group/single? Returned by date? - Written report, 10 15 pp. - Oral presentation 20 min 6 Grading In grading there are several components: The exam (max 30 p.) The compulsory exercises (together max 24 p.) Programming exercise (max 10 p.) An essay (max 10 p.) Verbal presentation (max 4 p.) The max points altogether is 54 p. Weekly exercises earn extra points (max 6 p.) Most probably the grade is decided as follows: points 30 34 40 44 48 grade 1 2 3 4 5 3
7 Material The textbook of the course is S. Russell, P. Norvig: Artificial Intelligence, A Modern Approach, Third ed., Pearson, 2010 There is no prepared material, the slides appear in the web as the lectures proceed http://www.cs.tut.fi/kurssit/ohj-2550/2556.html http://www.cs.tut.fi/~elomaa/teach/2556.html/ The exam is based on the lectures (i.e., not on the slides only) 8 Lecture Schedule 1. Introduction (I, 1) 1. Background 2. History of AI 3. Intelligent agents 2. Problem-solving (II, 2-3) 3. Knowledge, reasoning and planning (III, 4-6) 4. Uncertain knowledge and reasoning (V, 7-9) 5. Machine learning (VI, 10-11) 4
9 Weekly Exercises It is most advisable to take part in the weekly exercises The exercise questions appear in the web on Thu of previous week the latest Being ready to present one s own solution to a question publicly yields one mark. Each session has c. 5 questions you may gather altogether c. 5 12 = 60 marks Marks 30% 40% 50% 60% 70% 80% Extra points 1 2 3 4 5 6 10 TE vs. AI What is the lecturer s background on AI? His Ph.D. dissertation was on Machine Learning decision trees Algorithmic machine learning and data mining still are his main fields of research Publications, e.g., in Journal of Artificial Intelligence Research, Machine Learning, Data Mining and Knowledge Discovery, Journal of Machine Learning Research, Pattern Recognition, Chair of conferences on: Machine Learning, Principles of Data Mining and Knowledge Discovery, Foundations of Intelligent Systems, Discovery Science An old project concerned robotics Our robot was a Nomad Super Scout II equipped with sonars and a camera 5
11 1. INTRODUCTION Artificial intelligence is a wide and far-reaching concept. It also keeps changing over time Nowadays maybe more of a playground of philosophers and cognitive scientists From the point of view of computer science AI comprises of a set of more focused research fields that have already drifted quite far apart from each other The common goal in different subfields is to raise the intelligence of computers/machines I.e., to make the use of software easier As a result one gets ready-to-use software and theory charting out the boundaries of mechanical computation 12 1.1 What is AI? The only comparison to an intelligent machine that we are aware of is ourselves On the other hand, comparison to the human intelligence limits out other (better) alternatives Ideal intelligence is called rationality One can view intelligence from the point of view of though and behavior As combinations we get four distinct views to artificial intelligence: thinking humanly, acting humanly, thinking rationally, and acting rationally 6
13 Turing Test English mathematician Alan Turing proposed in 1950 the following criterion for the intelligence of a machine: a human interrogator cannot differentiate whether s/he is communicating with another human or a computer using text messages An example of a test of acting human-like In the so-called total Turing test the machine also has to be able to observe and manipulate its physical environment Time-limited Turing test competitions are organized annually The best performance against knowledgeable organizers is recorded by programs that try to fool the interrogator Human experts have the highest probability of being judged as non-humans 14 Cognitive modeling The study of human thought belongs to psychology and cognitive science Of course also neurophysiology and similar sciences are concerned with these topics There is a strong connection to AI when the chosen point of view is human thought The algorithms and data (knowledge) structures of thought / mind Allen Newell and Herbert Simon (1961): General Problem Solver (GPS) comparison between deduction steps and human deduction 7
15 Rationality Study of rational thought is essentially study of formal logic and logical deduction Methods that are based (only) on logics suffer from computational complexity issues and of the difficulty of expressing uncertain knowledge In the model of rational acting we examine agents An agent is something that acts A software agent is distinguished from a program e.g. by its autonomous control, capability to perceive its environment, ability to adapt to change, persistence over a prolonged time, ability to take on another s goals, and so forth 16 A rational agent works to reach the best possible outcome given its observations and knowledge Under uncertainty one aims at maximizing expectation over the outcome Rational performance on the long run may require one to perform seemingly irrationally for shorter periods In the real world total rationality is not usually possible (due to the lack of time) 8
17 1.3 The History of AI One can consider McCulloch ja Pitts (1943) to be the first AI publication It demonstrates how a network of simple computation units, neurons, can be used to compute the logical connectives (and, or, not, etc.) It is shown that all computable functions can be computed using a neural network It is suggested that these networks may be able to learn Hebb (1949) gives a simple updating rule for teaching neural networks Turing (1950) introduces his test, machine learning, genetic algorithms, and reinforcement learning In 1956 John McCarthy organized a meeting of researchers interested in the field, the name AI was invented 18 History (cont d 2/4) From the very beginning central universities have been CMU, MIT, and Stanford which are top universities in the field of AI even today McCarthy (1958) programming language Lisp In the 1950 s and 1960 s huge leaps forward were made in operating within microworlds (e.g., the blocks world) Also robotics went forward: e.g. Shakey from SRI (1969) As well research on neural networks (Widrow & Hoff, Rosenblatt s perceptron) Eventually it however became evident that the success within microworlds does not scale up as such It had been obtained without a deeper understanding of the target problem and by using computationally intensive methods 9
19 History (cont d 3/4) Neural networks were wiped out of computer science research for over a decade by Minsky and Papert s proof of the poor expressive power of the perceptron (xor function) In 1970 s expert systems were being developed, they gather the deep knowledge of one application field Expert systems gained a better expertise than human experts in many fields and they became the first commercial success story of AI Developing expert systems however turned out to be meticulous work that cannot really be made automatic Logic programming had its brightest time in the mid 1980 s Study of neural networks returned back to computer science research in the mid 1980 s 20 History (cont d 4/4) Also the raise of machine learning research dates back to the 1980 s The research of Bayesian networks also started at that time Maybe the second important commercial success due to the heavy influence of Microsoft Later on these topics have been studied under the label of data mining and knowledge discovery Agents are an important technology in many fields of computing A recent trend is also direction towards analytic research instead of using just ad hoc techniques Theoretical models of machine learning Well-founded methods of planning The new raise of game theory 10
21 1.4 The State of the Art Different activities in many subfields: Robotic vehicles: Driverless robotic cars are being developed in closed environments and more and more in daily traffic. Modern cars recognize speed limits, adapt to the traffic, take care of pedestrian safety, can park themselves, have intelligent light systems, wake up the driver, Speech recognition: Many devices and services nowadays understand spoken words (even dialogs) Autonomous planning and scheduling: E.g. space missions are tomorrow planned autonomously Game playing: Computers defeat human world champions in chess systematically and convincingly Spam fighting: Learning algorithms reliably filter away 80% or 90% of all messages saving us time for more important tasks 22 Logistics planning: E.g. military operations are helped by automated logistics planning and scheduling for transportation Robotics: Autonomous vacuum cleaners, lawn movers, toys, and special (hazardous) environment robots are common these days Machine translation: Translation programs based on statistics and machine learning are in ever increasing demand (in particular in EU) There are of course many other interesting AI applications some of them taking advantage of the Web 11
23 2 INTELLIGENT AGENTS An agent perceives its environment through sensors and acts upon the environment through actuators Our sensors include eyes, ears, nose, and other organs Our actuators include hands, legs, mouth, and other body parts The sensors of a robot (or a car) can include e.g. cameras, infrared and laser range finders and various motors as actuators A software agent receives keystrokes, file contents, and network packets as sensory inputs It acts on the environment by displaying on the screen, writing files, and sending network packets 24 We assume that every agent will perceive its own actions, but not always the effects In general, an agent s choice of action at any given instant can depend on the entire percept sequence observed to date The agent function maps any given percept sequence to an action The table of all possible input-output pairs of the function is a complete external characterization of the agent Of course such a table is infinite in most cases not applicable Internally the agent function for an artificial agent will be implemented by an agent program 12