Syllabus, Spring for: Agents, Games & Evolution OPIM 399, Section 1 Tuesdays, 5 7 p.m., 418 JMHH http://opim-sun.wharton.upenn.edu/ sok/teaching/age/s03/ Steven O. Kimbrough January 4, 2003 1 Brief Description Note: Agents, Games & Evolution is normally given under the course number OPIM 325. Because of scheduling issues, it is being given as OPIM 399, Section 1, this semester. Agents, Games & Evolution explores fundamentals strategy and strategic behavior. Strategic, or game-theoretic, topics arise throughout the social sciences. The topics include and we shall study trust, cooperation, marketrelated phenomena (including price equilibria and distribution of wealth), norms, conventions, commitment, coalition formation, and negotiation. They also include such applied matters as design of logistics systems and design of auctions. In addressing these topics we focus on the practical problem of finding effective strategies for agents in strategic situations (or games). Our method of exploration will thoroughly be experimental: we review and discuss experiments on the behavior of agents in strategic (or game-theoretic) situations. We will focus on the design and behavior of artificial agents in strategic (or game-theoretic) situations. We will be especially concerned with strategic contexts of commercial import, such as bargaining and repeated play. We shall dwell on effective agent learning techniques, including evolutionary methods and reinforcement learning. A main theme in the course is the inherent difficulty, even unknowability, of the problem of strategy acquisition. We will rely mainly on computational experiments (or simulations), in distinction to analytic mathematical methods, for studying strategy formation and strategic behavior (either by individuals or by groups). Much of the class work will be devoted to discussing and interpreting computational experiments that have been reported in the literature, or that can be undertaken with tools provided in class. In doing so, we draw upon the rapidly growing literature in 1
agent-based modeling and agent-based simulation. Computer programming is neither required nor discouraged for the course. Students completing the course can expect to come away with: Solid understanding of what is known and what is not known about the problem of designing procedures for strategic behavior, Familiarity with the principal methods, and results of applying those methods, for the modeling human agents and design of artificial agents in strategic contexts, and Deepened appreciation for complex, adaptive systems. 2 Instructor Professor Steven O. Kimbrough. Office hours: 9:00 12:00 on Tuesdays and by appointment. Contact information: University of Pennsylvania, The Wharton School, Jon M. Huntsman Hall (JMHH), Suite 500, Room 565, 3730 Walnut Street, Philadelphia, PA 19104-6340, 215-898-5133, kimbrough@wharton.upenn.edu. 3 Grading 30% Term project 25% Mid-term exam 25% Final exam 20% Class participation & assignments 4 Resources Course Home Page (HomePage): http://opim-sun.wharton.upenn.edu/ sok/teaching/age/s03/ ReadingsOnLine: http://opim-sun.wharton.upenn.edu/ sok/age/ Notes for Agents, Games, and Evolution by Steve Kimbrough (hereafter: AGEbook) will appear on the HomePage. 5 Class Schedule 1. (T:01/14) First class. Introduction to the course. Introductions. Overview of the course. Background concepts. Games, strategic contexts, interactive behavior. Normal/strategic form, extensive 2
form. Equilibrium outcomes. Pareto efficient outcomes. Constant sum, mixed motive, and coordination games. Interactive behavior and the social sciences. Assigned readings: AGEbook, Part I: chapter 1, A Society of Ideas, chapter 2, Computational Explanation, and chapter 3, Introducing Games. Recommended readings: Bulk Pack #4; J. Borwein, P. Borwein, R. Girgensohn, and S. Parnes (October 16, 1995). Experimental Mathematics: A Discussion (see ReadingsOnLine). 2. (T:01/21) Part I: Basics of Strategy-Centric Agents The problem of cooperation. Axelrod s experiments with strategies for iterated (repeated) prisoner s dilemma (IPD). Fundamental characteristics of IPD. Assigned readings: [1, Chapters 1 3, Appendices A & B]; AGEbook, Axelrod s IPD Tournaments. Behavior of plants and animals. Assigned readings: [1, Chapters 4 5] and Bulk Pack Reading #1; AGEbook, Notes: Maynard Smith & ESS and IPD Applications: War and Biology. 3. (T:01/28) Evolution of the social contract. Read: [19, Chapters 1 2]; AGEbook, Skyrms and the Social Contract. Evolution of the social contract. Read: [19, Chapters 3 4]; AGEbook, Skyrms and the Social Contract. 4. (T:02/04) Evolution of the social contract. Read: [19, Chapters 5 & Postscript]; AGEbook, Skyrms and the Social Contract. Beyond Prisoner s Dilemma Stag hunt and other games. Reading: Brian Skyrms, The Stag Hunt [20], BP#21. Recommended reading: Wolfgang Pesendorfer, Design Innovation and Fashion Cycles [13], BP#12; Brian Skyrms and Robin Pemantle, A Dynamic Model of Social Network Formation [21], BP#20. 3
Part II: Evolution, Rationality & Games 5. (T:02/11) Evolution and Equilibria Surprises and the Surprise Exam Assigned readings: AGEbook, Surprising Ramifications of the Surprise Exam. Assignment #1 Read the review article, Michael W. Macy and Robert Willer, From Factors to Actors: Computational Sociology and Agent-Based Modeling [10], BP#10. (Handed out; see also: http://opim.wharton.upenn. edu/~sok/papers/m/macy-willer-2002-ann-rev-sociology.pdf.) Pick one of the recent articles mentioned in the review paper, read it, write up a short description and critical assessment of it (5 pages max), and prepare a short presentation (5-10 minutes max) on it. The write up and the presentation are due in class on February 25, 2003. We ll devote much of that class period to your presentations and their discussion. 6. (T:02/18) Part III: Computational Principles & Applications Genetic algorithms and Evolution Programming Read: Bulk Pack Reading #11, [12, Chapter 1]; AGEbook, Introducing Genetic Algorithms. Undecidability, computational irreducibility, the halting problem. Assigned reading: AGEbook, Notes on Computability and Complexity. Background reading (recommended): AGEbook, Coding & Computation. 7. (T:02/25) Presentation of Assignment #1. Bargaining and coalition formation;genetic Programming etc. Assigned reading: Garett O. Dworman, Steven O. Kimbrough and James D. Laing, Bargaining by Artificial Agents in Two Coalition Games: A Study in Genetic Programming for Electronic Commerce [3], BP#3. Recommended reading: John R. Koza, Forrest H. Bennett III, David Andre, and Martin A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, [9, Chapter 2, pp. 17 66], BP#7. 4
8. (T:03/04) Midterm quiz. In class. The Decentralized Mindset. Read: Bulk Pack Reading #13. 9. (T:03/18) The Game of Life and cellular automata (CA). Read: Bulk Pack Readings #2, #17; AGEbook, Cellular Automata and the Game of Life. In-class demonstrations. Agent programming environments. (StarLogo, Swarm, etc.) and Evolutionary Game Theory. Read: Bulk Pack Reading #18. Spatialization Spatialized games. Read: Bulk Pack Readings #6, #19. 10. (T:03/25) Spatialized games. Read: Bulk Pack Reading #6. Societies of CA (Cellular Automata). Read: Growing Artificial Societies [4, Chapters I III]; AGEbook, GAS: Growing Artificial Societies. 11. (T:04/01) Societies of CA. Read: Growing Artificial Societies [4, Chapters IV VI]; Dhananjay K. Gode and Shyam Sunder, Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality [6], BP#5; AGEbook, GAS: Growing Artificial Societies. Identity-Oriented Agents Decentralization Again. Ecological Rationality. Identity-Oriented Agents. Read: Bulk Pack Reading #16; AGEbook, Identity-Oriented Agents. Recommended: #15. 12. (T:04/08) Dynamic programming. Read: Instructor s handout. 5
Reinforcement Learning. Reading: AGEbook, Reinforcement Learning ; Alvin E. Roth and Ido Erev, Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term [15], BP#14. Recommended reading: Bulk Pack Reading #8. See ReadingsOn- Line. 13. (T:04/15) LCS: Learning Classifier Systems Applications of Learning Classifier Systems 14. (T:04/22) Last class. Summing up. Dynamic, probing rationality. Darwin machines and Darwinian theories of mind. Reading: AGEbook, Dual Interpretations. 6 Required Books 1. Robert Axelrod, The Evolution of Cooperation, [1]. 2. Joshua Epstein and Rob Axtell, Growing Artificial Societies: Social Science from the Bottom Up, [4]. 3. B. Skyrms, Evolution of the Social Contract, [19]. 7 Bulk Pack Readings 1. John Maynard Smith, from Evolution and the Theory of Games [11, Chapters 1 3, pp. 1 39]. 2. Berlekamp, Conway, and Guy, from Winning Ways for Your Mathematical Plays, Volume 2 [2, Chapter 25, What Is Life?, pp. 817 850]. 3. Garett O. Dworman, Steven O. Kimbrough and James D. Laing, Bargaining by Artificial Agents in Two Coalition Games: A Study in Genetic Programming for Electronic Commerce [3]. 4. Gerd Gigerenzer and Reinhard Selten, Rethinking Rationality in [5]. 5. Dhananjay K. Gode and Shyam Sunder, Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality [6] 6. Patrick Grim, Gary Mar, and Paul St. Denis, The Philosophical Computer: Exploratory Essays in Philosophical Computer Modeling, [7, Chapters 4 5, pp. 155 236]. 6
7. John R. Koza, Forrest H. Bennett III, David Andre, and Martin A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving, [9, Chapter 2, pp. 17 66]. 8. Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore, Reinforcement Learning: A Survey. (http://www.cs.brown.edu/people/lpk/rlsurvey.ps). [8]. Available at ReadingsOnLine. 9. Epstein and Axtell, Growing Artificial Societies, [4]. 10. Michael W. Macy and Robert Willer, From Factors to Actors: Computational Sociology and Agent-Based Modeling [10] 11. Selections from M. Mitchell, An Introduction to Genetic Algorithms, [12], Chapter 1, Genetic Algorithms: An Overview, pages 1 33. 12. Wolfgang Pesendorfer, Design Innovation and Fashion Cycles [13] 13. Chapters 1, Foundations, and 4, Reflections, from Mitchel Resnick, Turtles, Termites, and Traffic Jams [14, Chapter 1, pp. 3 19; Chapter 4, pp. 119 144] 14. Alvin E. Roth and Ido Erev, Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term [15] 15. Thomas C. Schelling, Micromotives and Macrobehavior, [16]. 16. Reinhard Selten, What Is Bounded Rationality? in [17]. 17. Karl Sigmund, Chapter 2, Self-Replicating Automata and Artificial Life, pages 8 39 in [18]. 18. Karl Sigmund, Chapter 7, Evolutionary Game Theory, pages 154 179 in [18]. 19. Karl Sigmund, Chapter 8, Reciprocity and the Evolution of Cooperation, pages 180 206 in [18]. 20. Brian Skyrms and Robin Pemantle, A Dynamic Model of Social Network Formation [21]. 21. Brian Skyrms, The Stag Hunt [20]. 7
References [1] Robert Axelrod. The Evolution of Cooperation. Basic Books, Inc., New York, NY, 1984. [2] Elwyn R. Berlekamp, John H. Conway, and Richard K. Guy. Winning Ways for Your Mathematical Plays, volume 2: Games in Particular. Academic Press, New York, NY, 1982. [3] Garett O. Dworman, Steven O. Kimbrough, and James D. Laing. Bargaining by artificial agents in two coalition games: A study in genetic programming for electronic commerce. In John R. Koza, David E. Goldberg, David B. Fogel, and Rick L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Genetic Programming Conference, July 28-31, 1996, Stanford University, pages 54 62. The MIT Press, 1996. File: GP96 k08.doc. [4] Joshua M. Epstein and Robert Axtell. Growing Artificial Societies. The MIT Press, Cambridge, MA, 1996. [5] Gerd Gigerenzer and Reinhard Selten. Rethinking rationality. In Gerd Gigerenzer and Reinhard Selten, editors, Bounded Rationality: The Adaptive Toolbox, pages 1 12. MIT Press, Cambridge, MA, 2001. [6] Dhananjay K. Gode and Shyam Sunder. Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality. Journal of Political Economy, 101:119 37, 1993. [7] Patrick Grim, Gary Mar, and Paul St. Denis. The Philosophical Computer: Exploratory Essays in Philosophical Computer Modeling. The MIT Press, Cambridge, MA, 1998. [8] Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237 285, 1996. [9] John R. Koza, Forrest H. Bennett III, David Andre, and Martin A. Keane. Genetic Programming III: Darwinian Invention and Problem Solving. Morgan Kaufmann Publishers, San Francisco, CA, 1999. [10] Michael W. Macy and Robert Willer. From factors to actors: Computational sociology and agent-based modeling. Annual Review of Sociology, 28:143 66, 2002. [11] John Maynard Smith. Evolution and the Theory of Games. Cambridge Univesity Press, New York, NY, 1982. [12] Melanie Mitchell. An Introductin to Genetic Algorithms. The MIT Press, Cambridge, MA, 1996. 8
[13] Wolfgang Pesendorfer. Design innovation and fashion cycles. The American Economic Review, 84(4):771 92, 1995. [14] Mitchel Resnick. Turtles, Termites, and Traffic Jams. MIT Press, Cambridge, MA, 1994. [15] Alvin E. Roth and Ido Erev. Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8:164 212, 1995. [16] Thomas C. Schelling. Micromotives and Macrobehavior, chapter Micromotives and Macrobehavior, pages 9 43. W.W. Norton & Company, New York, NY, 1978. [17] Reinhard Selten. What is bounded rationality? In Gerd Gigerenzer and Reinhard Selten, editors, Bounded Rationality: The Adaptive Toolbox, pages 13 36. MIT Press, Cambridge, MA, 2001. [18] Karl Sigmund. Games of Life: Explorations in Ecology, Evolution and Behaviour. Oxford University Press, New York, NY, 1993. [19] Brian Skyrms. Evolution of the Social Contract. Cambridge University Press, New York NY, 1996. [20] Brian Skyrms. The stag hunt. Proceeding and Addresses of the American Philosophical Association, 2001. [21] Brian Skyrms and Robin Pemantle. A dynamic model of social network formation. Proceedings of the National Academy of Sciences, 97(16):9340 9346, August 1, 2000. $Id: age-syllabus.tex,v 1.7 2002/09/03 15:15:11 sok Exp $ 9