Advanced Multiprocessor Programming Vorbesprechung Jesper Larsson Träff traff@par.tuwien.ac.at Research Group Parallel Computing Faculty of Informatics, Institute of Information Systems Vienna University of Technology (TU Wien)
Crib Lecture: Monday, 10:00-12:00, Gusshausstrasse 25-29, EI 6 Eckert Exercises: Two batches, Thursday, 10:00-12:00, EI 6 Eckert Course HP (+TISS): http://www.par.tuwien.ac.at/teaching/2016s/184.726.psp
The facts and the problems Modern multi-core processors (2, 4,, 80 cores + multi/hyperthreading) do not really correspond to standard theoretical models (PRAM) Very, very difficult to program efficiently This course: Advanced programming techniques in theory and practice for modern multi-core processors (no GPUs): How to implement traditional constructs like locks and barriers efficiently How to program without locks and barriers: data structures and algorithms What work-stealing is and how to use it
Formalities VU (Lecture-Exercises-Project) 4.0 ECTS (=100 hours of work) Breakdown: Lecture 1.0 ECTS Exercises 1.0 ECTS Programming Project: 2.0 ECTS Participation MANDATORY, credit given based on Blackboard Exercises and Programming Project
Lecture: Monday, 10.00 (s.t!) - 12.00, EI 6 Gusshausstrasse 25-29 Thursday, 10.00 12.00, also EI 6: by need, for exercises and discussion Sprechstunde (Jesper Larsson Träff, Sascha Hunold): by appointment Email: traff@par.tuwien.ac.at, hunold@par.tuwien.ac.at
Sign-up required (deadline 31.3, TISS) Sign-out if you don t follow the lecture Theory exercises should be done individually (discussions encouraged ) Project in groups of 2 (sign-up required) Get account via TUWEL: 14-20.3 (will be enabled this week)
Topics, Goals Basic understanding of principles and practice of thread-based shared-memory multiprocessor programming Principles/theory: Synchronization and coordination mechanisms Scope and limitations Correctness: safety and liveness Practice: Implementation of basic synchronization mechanisms Fundamental (lock- and wait-free) data structures Memory models PHEET framework, CilkPlus, C++ threads Supporting higher-level shared memory programming models: Task parallel models by work-stealing (Transactional memory)
Literature/Material Book: Maurice Herlihy (Brown), Nir Shavit (Tel Aviv): The Art of Multiprocessor Programming. Morgan Kaufmann Publishers, 2008, revised 1 st edition, 2012 Lecture slides, additional papers Recommended: buy it! Course material: http://www.par.tuwien.ac.at/teaching/2016s/184.726.psp
Other good books Taubenfeld: Synchronization Algorithms and Concurrent Programming. Pearson/Prentice Hall 2006 Kshemkalyani,Singhal: Distributed computing: principles, algorithms, and systems Cambridge University Press, 2011
Raynal: Concurrent programming: Algorithms, Principles, and Foundations. Springer, 2013
Approx. Coverage Chapters 1-5 (6), Chapters 7, 9, 10, 11, (12?), 13-16, (17?) Work-stealing and memory models from other sources Prerequisites: Introduction to Parallel Computing Algorithms and data structures C/C++ (or Java) programming Possible follow-up: Parallel Algorithms HPC Distributed Algorithms (Ulrich Schmid)
Exercises/Project Theoretical exercises from book, hand-in and discussion/presentation on blackboard Two slots Small programming project: Implementation and benchmarking (comparison) of lock-free data structure(s) and other material from the lectures What we may do: use the PHEET C++ work-stealing framework ( Martin Wimmer) as surrounding framework with paradigm application, plug in own data structure and compare But, it will also be possible to do project in CilkPlus, pthreads, Java, C++
Exercises: 2 batches, hand-in and blackboard (April&May) 21.4 (Thursday) 12.5 (Thursday) Project is done in groups of 2 Project: 25.4 (Monday): Project topic presentation (by me) 2.6 (Thursday): Project status presentation (by you: each group gives a 10-15 minute overview of what it is doing) 1.7: Project hand-in (fixed deadline, no extension) EXAM: Early July (4.-8.7) or August
System Possible to start developing on own PC/laptop (no lab access) Benchmarking/testing: Saturn shared-memory node at TUWien: 4xAMD magny cours 12-core Opteron 6168 processors 128GByte main memory, 1.9GHz, total number of cores 48 Possibly also: mars.par.tuwien.ac.at Name: saturn.par.tuwien.ac.at More later (get account via TUWEL from 14.3 till 20.3)
Grading/participation Attending lectures and exercises Active participation Solving the exercises, presentation on the blackboard (theoretical exercises, hand-in of practical programming exercise) Group examination for project part NOTE: You only learn by doing exercises and project by yourself. Copying will result in grade 6 Discussion with other groups encouraged, but hand in your own solution Don t forget: EVALUATE THE COURSE by end of semester (TISS)
Project hand-in: Short description of problem, your solution Some argument for correctness, testing procedure The required tests/benchmark comparisons (plots, tables) Both correctness and performance are important! Grade weighting: ¼ for exercises, ½ for project, ¼ for exam Solving in group: Active collaboration, 2*100%, NOT 2*50% Both members get same grade (unless blatantly different) Both members must understand all aspects of solutions
Plan 7.3.: Vorbesprechung 14.3.: Intro, Mutual exclusion problem and solutions. 21.3, 28.3: EASTER, no lectures 4.4.: Mutual exclusion (con't), an impossibility result, constructions with registers. 11.4.: Relative power of synchronization operations, correctness 18.4.: Universality. 21.4: Exercises Batch 1 25.4.: Practical lock implementations. Project topics 2.5.: Data structures (I): List-based sets 9.5.: Data structures (II): Queues, Stacks 12.5: Batch 2 16.5.: WHITSUN, no lecture 23.5.: No lecture (project work) 30.5: Data structures (III): Skiplist. 2.6: Project status 6.6: Data structures (IV): Skiplist, Combining network 13.6.: Memory consistency models 20.6.: Work-stealing theory 27.6: TBA
Follow-up Project (12.0 ECTS) Seminar in, WS16, Parallel Algorithms (WS16: VU, 3.0 ECTS) High Performance Computing (: VU, 3.0 ECTS) Master s Thesis (30.0 ECTS) Talks in the group everybody is welcome, see http://www.par.tuwien.ac.at/talks-guests.psp