Advanced Multiprocessor Programming

Size: px

Start display at page:

Download "Advanced Multiprocessor Programming"

Annice Anderson
6 years ago
Views:

at Research Group Parallel Computing Faculty of

1 Advanced Multiprocessor Programming Vorbesprechung Jesper Larsson Träff Research Group Parallel Computing Faculty of Informatics, Institute of Information Systems Vienna University of Technology (TU Wien)

2 Crib Lecture: Monday, 10:00-12:00, Gusshausstrasse 25-29, EI 6 Eckert Exercises: Two batches, Thursday, 10:00-12:00, EI 6 Eckert Course HP (+TISS):

3 The facts and the problems Modern multi-core processors (2, 4,, 80 cores + multi/hyperthreading) do not really correspond to standard theoretical models (PRAM) Very, very difficult to program efficiently This course: Advanced programming techniques in theory and practice for modern multi-core processors (no GPUs): How to implement traditional constructs like locks and barriers efficiently How to program without locks and barriers: data structures and algorithms What work-stealing is and how to use it

4 Formalities VU (Lecture-Exercises-Project) 4.0 ECTS (=100 hours of work) Breakdown: Lecture 1.0 ECTS Exercises 1.0 ECTS Programming Project: 2.0 ECTS Participation MANDATORY, credit given based on Blackboard Exercises and Programming Project

5 Lecture: Monday, (s.t!) , EI 6 Gusshausstrasse Thursday, , also EI 6: by need, for exercises and discussion Sprechstunde (Jesper Larsson Träff, Sascha Hunold): by appointment traff@par.tuwien.ac.at, hunold@par.tuwien.ac.at

6 Sign-up required (deadline 31.3, TISS) Sign-out if you don t follow the lecture Theory exercises should be done individually (discussions encouraged ) Project in groups of 2 (sign-up required) Get account via TUWEL: (will be enabled this week)

7 Topics, Goals Basic understanding of principles and practice of thread-based shared-memory multiprocessor programming Principles/theory: Synchronization and coordination mechanisms Scope and limitations Correctness: safety and liveness Practice: Implementation of basic synchronization mechanisms Fundamental (lock- and wait-free) data structures Memory models PHEET framework, CilkPlus, C++ threads Supporting higher-level shared memory programming models: Task parallel models by work-stealing (Transactional memory)

8 Literature/Material Book: Maurice Herlihy (Brown), Nir Shavit (Tel Aviv): The Art of Multiprocessor Programming. Morgan Kaufmann Publishers, 2008, revised 1 st edition, 2012 Lecture slides, additional papers Recommended: buy it! Course material:

9 Other good books Taubenfeld: Synchronization Algorithms and Concurrent Programming. Pearson/Prentice Hall 2006 Kshemkalyani,Singhal: Distributed computing: principles, algorithms, and systems Cambridge University Press, 2011

10 Raynal: Concurrent programming: Algorithms, Principles, and Foundations. Springer, 2013

11 Approx. Coverage Chapters 1-5 (6), Chapters 7, 9, 10, 11, (12?), 13-16, (17?) Work-stealing and memory models from other sources Prerequisites: Introduction to Parallel Computing Algorithms and data structures C/C++ (or Java) programming Possible follow-up: Parallel Algorithms HPC Distributed Algorithms (Ulrich Schmid)

12 Exercises/Project Theoretical exercises from book, hand-in and discussion/presentation on blackboard Two slots Small programming project: Implementation and benchmarking (comparison) of lock-free data structure(s) and other material from the lectures What we may do: use the PHEET C++ work-stealing framework ( Martin Wimmer) as surrounding framework with paradigm application, plug in own data structure and compare But, it will also be possible to do project in CilkPlus, pthreads, Java, C++

13 Exercises: 2 batches, hand-in and blackboard (April&May) 21.4 (Thursday) 12.5 (Thursday) Project is done in groups of 2 Project: 25.4 (Monday): Project topic presentation (by me) 2.6 (Thursday): Project status presentation (by you: each group gives a minute overview of what it is doing) 1.7: Project hand-in (fixed deadline, no extension) EXAM: Early July (4.-8.7) or August

System Possible to start developing on own PC/laptop (no lab access) Benchmarking/testing: Saturn shared-memory node at TUWien: 4xAMD magny cours 12-core Opteron 6168

14 System Possible to start developing on own PC/laptop (no lab access) Benchmarking/testing: Saturn shared-memory node at TUWien: 4xAMD magny cours 12-core Opteron 6168 processors 128GByte main memory, 1.9GHz, total number of cores 48 Possibly also: mars.par.tuwien.ac.at Name: saturn.par.tuwien.ac.at More later (get account via TUWEL from 14.3 till 20.3)

15 Grading/participation Attending lectures and exercises Active participation Solving the exercises, presentation on the blackboard (theoretical exercises, hand-in of practical programming exercise) Group examination for project part NOTE: You only learn by doing exercises and project by yourself. Copying will result in grade 6 Discussion with other groups encouraged, but hand in your own solution Don t forget: EVALUATE THE COURSE by end of semester (TISS)

16 Project hand-in: Short description of problem, your solution Some argument for correctness, testing procedure The required tests/benchmark comparisons (plots, tables) Both correctness and performance are important! Grade weighting: ¼ for exercises, ½ for project, ¼ for exam Solving in group: Active collaboration, 2*100%, NOT 2*50% Both members get same grade (unless blatantly different) Both members must understand all aspects of solutions

17 Plan 7.3.: Vorbesprechung 14.3.: Intro, Mutual exclusion problem and solutions. 21.3, 28.3: EASTER, no lectures 4.4.: Mutual exclusion (con't), an impossibility result, constructions with registers : Relative power of synchronization operations, correctness 18.4.: Universality. 21.4: Exercises Batch : Practical lock implementations. Project topics 2.5.: Data structures (I): List-based sets 9.5.: Data structures (II): Queues, Stacks 12.5: Batch : WHITSUN, no lecture 23.5.: No lecture (project work) 30.5: Data structures (III): Skiplist. 2.6: Project status 6.6: Data structures (IV): Skiplist, Combining network 13.6.: Memory consistency models 20.6.: Work-stealing theory 27.6: TBA

18 Follow-up Project (12.0 ECTS) Seminar in, WS16, Parallel Algorithms (WS16: VU, 3.0 ECTS) High Performance Computing (: VU, 3.0 ECTS) Master s Thesis (30.0 ECTS) Talks in the group everybody is welcome, see

Advanced Multiprocessor Programming

Advanced Multiprocessor Programming Vorbesprechung Jesper Larsson Träff, Sascha Hunold traff@par. Research Group Parallel Computing Faculty of Informatics, Institute of Information Systems Vienna University