Series Editors Editorial Board
ABC
Alexander Shen Laboratoire d Informatique Fondamentale de Marseille (LIF) CNRS, Université delaméditerranée, Université deprovence CMI 39 Rue Joliot-Curie 13453 Marseille Cedex 13 France alexander.shen@lif.univ-mrs.fr and Russian Academy of Sciences Institute for Information Transmission Problems Bolshoy Karetny per. 19 Moscow, GSP-4, 127994 Russia Series Editors Jonathan M. Borwein, FRSC Professor Laureate Director Centre for Computer Assisted Research Mathematics and its Applications, CARMA School of Mathematical & Physical Sciences University of Newcastle Callaghan NSW 2308 Australia Jonathan.Borwein@newcastle.edu.au Helge Holden Department of Mathematical Sciences Norwegian University of Science and Technology Alfred Getz vei 1 NO-7491 Trondheim Norway holden@math.ntnu.no ISSN 1867-5506 e-issn 1867-5514 ISBN 978-1-4419-1747-8 e-isbn 978-1-4419-1748-5 DOI 10.1007/978-1-4419-1748-5 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2009942420 Mathematics Subject Classification (2000): 65K05, 65Yxx, 90Cxx, 68-01, 68W40 1st edition: c Birkhäuser 1997 Reprint of 1st edition in series: Modern Birkhäuser Classics c Birkhäuser 2008 2nd edition: c Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
To the memory of Anna Pogossiants
Contents Preface to the second edition....................................... xi 1 Variables, expressions, assignments............................. 1 1.1 Problems without arrays..................................... 1 1.2 Arrays.................................................... 15 1.3 Inductive functions......................................... 28 2 Generation of combinatorial objects............................ 33 2.1 Sequences................................................. 33 2.2 Permutations.............................................. 34 2.3 Subsets................................................... 35 2.4 Partitions................................................. 37 2.5 Gray codes and similar problems.............................. 39 2.6 Some remarks............................................. 44 2.7 Counting.................................................. 46 3 Tree traversal (backtracking).................................. 49 3.1 Queens not attacking each other: position tree................... 49 3.2 Tree traversal.............................................. 50 3.3 Queens: position tree implementation.......................... 56 3.4 Backtracking in other problems............................... 59 4 Sorting..................................................... 61 4.1 Quadratic algorithms........................................ 61 4.2 Sorting in n log n operations................................. 62 4.3 Applications of sorting...................................... 69 4.4 Lower bound for the number of comparisons.................... 70 4.5 Problems related to sorting................................... 72 vii
viii Contents 5 Finite-state algorithms in text processing......................... 77 5.1 Compound symbols, comments, etc............................ 77 5.2 Numbers input............................................. 79 6 Data types.................................................. 83 6.1 Stacks.................................................... 83 6.2 Queues................................................... 89 6.3 Sets...................................................... 97 6.4 Priority queues............................................. 100 7 Recursion................................................... 103 7.1 Examples................................................. 103 7.2 Trees: recursive processing.................................. 106 7.3 The generation of combinatorial objects; search................. 109 7.4 Other applications of recursion............................... 113 8 Recursive and non-recursive programs.......................... 119 8.1 Table of values (dynamic programming)....................... 119 8.2 Stack of postponed tasks..................................... 123 8.3 Difficult cases............................................. 126 9 Graph algorithms............................................ 129 9.1 Shortest paths.............................................. 129 9.2 Connected components, breadth- and depth-first search........... 133 10 Pattern matching............................................. 139 10.1 Simple example............................................ 139 10.2 Repetitions in the pattern.................................... 142 10.3 Auxiliary lemmas.......................................... 143 10.4 Knuth Morris Pratt algorithm................................ 144 10.5 Boyer Moore algorithm..................................... 146 10.6 Rabin Karp algorithm...................................... 148 10.7 Automata and more complicated patterns....................... 149 10.8 Suffix trees................................................ 156 11 Games analysis.............................................. 167 11.1 Game examples............................................ 167 11.2 Game cost................................................. 169 11.3 Computing the game cost by backtracking...................... 176 11.4 Alpha-beta pruning......................................... 178 11.5 A retrospective analysis..................................... 182
Contents ix 12 Optimal coding.............................................. 185 12.1 Codes.................................................... 185 12.2 The Kraft McMillan inequality............................... 186 12.3 Huffman code............................................. 189 12.4 Shannon Fano code........................................ 191 13 Set representation. Hashing.................................... 195 13.1 Hashing with open addressing................................ 195 13.2 Hashing using lists......................................... 198 14 Sets, trees, and balanced trees.................................. 203 14.1 Set representation using trees................................. 203 14.2 Balanced trees............................................. 210 15 Context-free grammars....................................... 221 15.1 General parsing algorithm................................... 221 15.2 Recursive-descent parsing................................... 226 15.3 Parsing algorithm for LL(1)-grammars......................... 236 16 Left-to-right parsing (LR)..................................... 243 16.1 LR-processes.............................................. 243 16.2 LR(0)-grammars........................................... 248 16.3 SLR(1)-grammars.......................................... 252 16.4 LR(1)-grammars, LALR(1)-grammars......................... 253 16.5 General remarks about parsing algorithms...................... 256 Further reading.................................................. 257 Appendix: C and Pascal examples.................................. 259 Index........................................................... 267
Preface to the Second Edition Somebody once said that one may prove the correctness of an algorithm, but not of a program. One of the main goals of this book is to convince the reader that things are not so bad. A well-known programmer, C.A.R. Hoare, said that the beauty of a program is not an additional benefit but a criterion that separates success from failure. If, while solving problems in this book, you come to appreciate the beauty of a well-written program with each part in its correct place, the author s goal will have been reached. Theoretically this book can be used to study programming without a computer: one could write (correct) programs with pencil and paper. But in practice the ability to run the programs is a challenge and a reward that makes programming a fun. We have utilized the problem-solution format. Some chapters are collections of problems having a common topic, while others are devoted to one specific algorithm (e.g., chapter 16 covers LR(1)-parsing). The chapters are more or less independent, but the concluding chapters are more difficult. Chapters 1 7 cover material usually included in undergraduate courses while chapters 15 16 are more appropriate for a graduate compiler course. In each chapter we have tried to give problems at different levels starting with easy exercises. Problems are usually provided with solutions, answers or hints. However, we strongly recommend to read the solution only after the reader makes a good faith attempt to solve it independently. The book is restricted to micro-programming leaving aside another very important topic: how to split the program into a manageable parts with nice interfaces between them. (Probably this can be learned only by reading and modifying rather large programs.) Pascal is used as a programming language; though being outdated, it is reasonably clear, so the readers familiar with any other procedural language (C, Modula, Oberon, etc.) will encounter no difficulties. For the reader s convenience, a short appendix is added that lists basic differences between Pascal and C. It is intended to help the reader who knows C to understand the program notation in the book (but cannot replace textbooks on C). xi
xii Preface to the second edition Most of the problems, of course, are well known. References are rare, but absence of references does not mean that the problem or algorithm is new. However, we hope that in some cases the algorithm or the proof is explained better than what is found in other sources. This book is addressed both to the ambitious student who wants to test and improve his/her skills and to the instructor looking for problems for his/her class. I thank all the people I met while teaching programming (first of all, my former students from 57th school and A.G. Kushnirenko, who was my programming teacher) and all readers that sent me corrections for the preliminary versions of this book (especially Yu.V. Matijasevich). I also thank American Mathematical Society (former Soviet Union aid fund), International Science Foundation, Open Society Foundation, MIT, University of Bordeaux, Bonn University, the Rosenbaum Foundation, INTAS, University of Provence, CNRS, Institute of Problems of Information Transmission and even the Russian government for support during writing this book. I thank Ann Kostant, Elizabeth Loew and the other nice people at Birkhäuser and Springer for their help. Tom Scavo did a great job correcting my English (as well as several other errors) but in no case should he be blamed for the remaining mistakes. Peter Panov helped to prepare the second edition by translating new material in chapters 10 12 and language editing. The Russian version of this book is freely distributable as a TEX source and camera-ready copy; please look at ftp://ftp.mccme.ru/users/shen/progbook and/or contact the author (e-mail addresses: shen@landau.ac.ru, shen@mccme.ru, sasha.shen@gmail.com, alexander.shen@lif.univ-mrs.fr) for details. I d be grateful if bug reports could be sent to the same addresses. Alexander Shen September, 2009