Infrastructure Issues Related to Theory of Computing Research Faith Fich, University of Toronto Theory of Computing is a eld of Computer Science that uses mathematical techniques to understand the nature of computation and to design and analyze algorithms for important and fundamental problems. It provides important new ways of thinking about computation and lasting insights that are applicable to a wide variety of systems. There are a range of dierent research activities that occur within this eld, including developing models (and ways to quantify the eciency of algorithms in those models), determining the relationships among dierent models, designing new (provably correct) algorithms, analyzing the eciency of various algorithms, proving lower bounds and impossibility, and explaining experimental results. Some researchers are better at posing interesting questions or opening up new directions for research; others are better at answering questions and developing new techniques to do so. In this paper, I wish to discuss a number of issues related to the support of research in Theory of Computing, including funding, evaluation of research, communication, interaction with other research areas, education, and employment. It is directed primarily to researchers in Theory of Computing and is meant to complement the Infrastructure Section of the committee report \Strategic Directions for Research in Theory of Computing", which is directed primarily to people outside our community. I will examine some things our community has done well, some recent innovations, and some suggestions to enhance our research environment. Finally, I will address the relationship of Theory of Computing with experimental computing and algorithmic engineering. Funding and Evaluation of Research Theory of Computing is one of the oldest and most established elds of computer science, so there is a high degree of consensus about the criteria for good research in Theory of Computing. See, for example, [7]. These criteria value many dierent kinds of research activities (as described in Section IIB of the committee report). Most researchers in Theory of Computing believe that funding of research within Theory of Computing, as well as interdisciplinary research, should be based primarily on the quality of the work, as judged by peer review. It is important that the peer review process reect the community-wide consensus on high quality. For example, applying signicantly dierent standards to papers on certain topics or using certain approaches when choosing the papers for a conference can substantially lower the reputation of the conference and lead to dissension within our community. The diversity of thought and approach within Theory of Computing makes it healthy. Attempts to channel research in a few selected directions through funding policies and criteria for hiring, tenure, and promotion, are harmful. For example, at NSF in the United States, there is a program director for Theory of Computing, who gets a large amount of additional funds from directors of application areas to support theoretical research in these areas. This put pressure on researchers to choose these areas and to do more applied work within these areas. One suggestion to address this problem is to directly fund Theory of Computing as a whole, through one program director. This requires that signicantly more money be directly allocated to Theory. Strong arguments for this based on the quality of our science and the important longterm impact of our eld, such as those described in the committee report and in [4] must be made. Then the funding of particular proposals within Theory of Computing should be based mostly on the quality of their science. In eect, this is saying that impact should be much more important in determining long-term policy than in determining short-term policy. 1
Another approach is to have a small panel of researchers, consisting of theoreticians, experimental scientists, and engineers be responsible for making funding decisions for the entire spectrum of proposals within an application area, including theory. This would make it easier to fairly judge proposals that involve both theory and experimentation and perhaps increase cooperation between theoreticians, experimental scientists, and engineers doing research in that area. In sciences where funding is divided according to subject matter, there is more competition for funding between areas than between theoreticians and experimentalists. However, diculties would arise for researchers engaged in theoretical work involving more than one application area, involving an application area outside of Computer Science, or involving core theory work that has no direct connections to a particular application area. In Canada, funding does not steer research to as great an extent. It may be due in part to the fact that NSERC grant applications from all areas of Computer Science are evaluated by one committee, composed of respected researchers from academia and industry. Unfortunately, this approach does not seem easily scaled to handle the larger number of researchers in the United States. When discussing the future of the Theory of Computing research, we must clearly distinguish between scientic issues and external forces, such as national funding policies and employment opportunities, aecting researchers. Only the former should inuence academic research. Finally, just because a question or subarea is new, doesn't make it more important or worthy of study. It is stable, long term funding that really facilitates progress. Communication and Interaction with Other Research Areas It is important to communicate the many successes of Theory of Computing and our excitement about them to our colleagues in Computer Science, to other scientists, to leaders in our communities and governments, and to the general public. In this way, we increase recognition and support for Theory of Computing, improve research funding (perhaps from sources that have not traditionally supported us), improve employment opportunities for our students, and attract new people into our eld. Visits to elementary and secondary schools, workshops for students, public lectures, expository articles, and press releases help to publicize the fact that we are working on important questions and we are making good progress. (Some interesting ways of presenting Theory of Computing topics to elementary school students appear in [3].) SIGACT should take a proactive role advertising the contributions of Theory of Computing, for example, by oering prizes for the best expository articles and lectures and making them available electronically. When researchers in Theory of Computing interact with researchers in other parts of Computer Science, in Computer Engineering, and in other disciplines, they enrich one another. This may involve identifying and explaining relevant research results, using expertise in one eld to help research in another eld, or working together on specic interdisciplinary projects. New problems and even new research directions can be inspired by such interaction. A new award, the Kannelakis award for theoretical contributions with impact on practice, was recently established by ACM. Textbooks that address the theoretical, experimental, and engineering aspects of a subject, written jointly by researchers from the dierent communities, are very valuable in helping to establish a common point of reference. To make our results more easily accessible to researchers outside of Theory of Computing, we should encourage the writing of survey papers describing the contributions of Theory of Computing to various application areas, explaining what certain results in Theory of Computing say about these areas and briey outlining techniques we have developed that may be useful to them. 2
These papers should be written in the terminology and notation of the specic application areas and should be published in journals typically read by researchers in those areas. To emphasize the importance of these papers and point out specic good examples, the Theory of Computing community should award prizes to the best papers in this category. I do not think that having each research paper include a broadly accessible expository introduction is a good way to communicate the practical applications, in contrast to [1]. Besides wasting paper, this distributes the information, making it harder to access. Furthermore, the applicability of the results may be misrepresented or overstated, although vigilant refereeing can address this problem. Conversely, we should encourage papers in theoretically oriented journals (or at particular websites) describing issues and open problems arising from application areas and suggesting new research directions. Ideally, these papers would provide enough background to help theory researchers understand what directions are likely to be useful. Brief descriptions of current open problems, available electronically or in SIGACT News, that are presented in a clear and accessible way and that reference a few well chosen articles from the literature of the application area are also a good way to interest Theory of Computing researchers in such problems. Plenary talks by prominent researchers are particularly eective at instilling enthusiasm about particular research questions and areas. Conferences in specic subject areas (for example, parallel computation or computational biology) that include a broad spectrum of researchers encourage dialogue and interaction. Tutorials for nonspecialists and panel discussions are also very helpful. When joint sponsorship by the dierent communities is not feasible, scheduling conferences together provides some of the same benets. The two Federated Computing Research Conferences (in 1993 and 1996) have been very successful. Holding multiple conferences at the same location is important to Theory of Computing researchers with limited research funding. Other ways of reducing the costs of attending conferences should be examined. For example, lunches and banquets could be made optional or proceedings could be published electronically. Less expensive journals (including electronic journals) help to keep the Theory of Computing community inclusive. Up-to-date databases and bibliographies of conference and journal papers, are important and widely used research tools in Theory of Computing. See, for example, the Computer Science Bibliography Glimpse Server and the Hypertext Bibliography Project. Education and Employment Because Theory of Computing has signicant breadth as well as depth, students in Theory of Computing need a broad education in core theory, so that they become familiar with a wide variety of techniques and see connections between results in dierent parts of the eld. The STOC and FOCS conferences are forums for some of the best work throughout Theory of Computing. They promote breadth among Theory of Computing researchers and help to keep the community together. Survey talks discussing recent advances in a particular area have been well received at these conferences. In some cases, simultaneous submission of research papers to STOC or FOCS and a specialty conference is allowed. Although it is unnecessary to publish the associated paper in both proceedings, perhaps two dierent talks, one geared to specialists and one geared to a broader audience, could be presented at the two conferences. This may help to maintain the role of STOC and FOCS. Many interesting and high quality graduate courses have been developed in a wide range of topics within Theory of Computing. Lecture notes and interesting assignment questions (and solutions) for graduate and undergraduate courses are informally shared among certain profes- 3
sors at dierent universities. It would be nice if SIGACT could establish a (possibly electronic) mechanism for making these instructional aids more widely available to interested faculty and, at the same time, provide recognition for high quality contributions. Besides being an aid to faculty members, these provide a way for researchers to quickly get up to speed in a new research area. The education column and reviews of textbooks for theory courses that appear in SIGACT News are also very helpful. In addition, students in Theory of Computing need exposure to other parts of Computer Science: to understand the areas in which their work might be applied, learn the techniques that are typically used, and see the contributions and impact that Theory of Computing makes. Opportunities for students from dierent parts of Computer Science to talk and interact with one another should be provided. Industrial internships, summer jobs, and visits can be very benecial for Theory of Computing students and should be encouraged, when appropriate. For some faculty members in Theory of Computing, industrial visits or engaging in experimental or applicationspecic research while on sabbatical can stimulate new and interesting theoretical work. There are many inuential people in industry who say that they want to hire people trained in Theory of Computing because of the critical skills this training has given them. The ability to quickly understand a new application and ask the right questions is, in the long run, more important than having done a thesis involving large amounts of implementation. These facts should be more widely known by students, our colleagues in other parts of Computer Science, department chairs, deans, and other inuential people in industry. Perhaps a well-written report or a SIGACT News article by an appropriate industry person would help. Another suggestion is to compile a list of people whose PhD theses were in Theory of Computing and who have since made signicant, practical contributions to Computer Science or an application area. It would also be useful to gather statistics on the number of Theory of Computing graduates who take industrial employment and the kinds of work they do. The Relationship with Experimental Computer Science and Algorithmic Engineering Experimental computer science and algorithmic engineering are newly emerging, legitimate elds of Computer Science research. Many Theory of Computing researchers have made signicant contributions to these elds. However, this fact does not make these elds into subelds of Theory of Computing! Many other elds in Computer Science arose from within Theory of Computing. For example, Compiler Design began as part of Formal Language Theory. Dierences in research methodology, the kinds of questions being addressed, and the criteria applied to judge the quality of research make it natural for separation to occur. An individual's research programme (and even particular research papers) can still combine elements from more than one eld. Certainly, Theory of Computing researchers can gain valuable insight into problems through the use of experimentation. However, there is not yet consensus whether using only experimental techniques (for example, to understand the behaviour of an algorithm or determine which of two algorithms is faster on a particular set of data) is Theory of Computing research. One reason is that not all theoreticians are familiar with how experimental work contributes to an understanding of computation. Another reason is that, in new areas, especially interdisciplinary ones, it takes time to establish appropriate measures of quality. Progress is being made: there are a number of recent papers [5, 2, 6] that discuss standards for and evaluation of work that uses experimental techniques to understand the behaviour of specic algorithms. It would also be valuable to have an invited plenary lecture at the next FOCS or STOC conference address this 4
issue by giving examples of good experimental research with consequences of a theoretical nature, describing general techniques and methodologies that have been developed as part of this work, and explaining what experimental work should NOT be considered a contribution to Theory of Computing, because of aim, content, conclusions, or poor quality. Various specialty conferences, such as SODA and SPAA, are accepting some experimental papers. The Journal of Experimental Algorithmics was recently established. With more experience, it will be easier to decide whether this work belongs within Theory of Computing or is a separate research area. If experimental algorithmics is to be considered part of Theory of Computing, substantially more funding must be provided to Theory of Computing to support the broader range of activities and the increased cost of such research. It must also be emphasized that it is not necessary nor desirable that all Theory of Computing researchers engage in experimental work. Rather, researchers should be encouraged to continue to contribute in the ways that each does best. Acknowledgements Paul Beame, Al Borodin, Steve Cook, Vassos Hadzilacos, Nick Pippenger, Charlie Racko, Avi Wigderson, and the other members of the working-group on Theory of Computing contributed to this report through valuable discussions and feedback on various drafts. References [1] A. Aho, D. Johnson, R. Karp, R. Kosaraju, C. McGeoch, C. Papadimitriou, and P. Pevzner, Theory of Computing: Goals and Directions, manuscript, 1996. [2] W. Barr, B. Golden, J. Kelly, M. Resende, and W. Stewart, Designing and Reporting on Computational Experiments with Heuristic Methods, J. Heuristics, vol. 1, no. 1, 1995, pages 9{32. [3] M. Fellows, reference not yet known. [4] O. Goldreich and A. Wigderson, Theory of Computing: A Scientic Perspective, manuscript, 1996. [5] D. Johnson, How to do Experiments, manuscript, 1996. [6] C. McGeoch, Towards an Experimental Method for Algorithm Simulation, INFORMS J. Computing, vol. 8, no. 1, 1996, pages 1{15. [7] I. Parberry, Referee's Guide, SIGACT News, vol. 20, no. 4, Fall 1989 (whole number 73), pages 92{109. 5