Introduction to Parallel Computing

Abstract

This monograph is an overview of practical parallel computing and starts with the basic principles and rules which will enable the reader to design efficient parallel programs for solving various computational problems on the state-of-the-art computing platforms. The book too was written in parallel. The opening Chap. 1: “Why do we need Parallel Programming” has been shaped by all of us during instant communication immediately after the idea of writing such a book had cropped up. In fact, the first chapter was an important motivation for our joint work. We spared no effort in incorporating of our teaching experience into this book. The book consists of three parts: Foundations, Programming, and Engineering, each with a specific focus: • Part I, Foundations, provides the motivation for embarking on a study of parallel computation (Chap. 1) and an introduction to parallel computing (Chap. 2) that covers parallel computer systems, the role of communication, complexity of parallel problem-solving, and the associated principles and laws. • Part II, Programming, first discusses shared memory platforms and OpenMP (Chap. 3), then proceeds to message passing library (Chap. 4), and finally to massively parallel processors (Chap. 5). Each chapter describes the methodology and practical examples for immediate work on a personal computer. • Part III, Engineering, illustrates parallel solving of computational problems on three selected problems from three fields: Computing the number p (Chap. 6) from mathematics, Solving the heat equation (Chap. 7) from physics, and Seam carving (Chap. 8) from computer science. The book concludes with some final remarks and perspectives (Chap. 9). To enable readers to immediately start gaining practice in parallel computing, Appendix A provides hints for making a personal computer ready to execute parallel programs under Linux, macOS, and MS Windows. Specific contributions of the authors are as follows: • Roman Trobec started the idea of writing a practical textbook, useful for students and programmers on a basic and advanced levels. He has contributed Chap. 4: “MPI Processes and Messaging”, Chap. 9: “Final Remarks and Perspectives”, and to chapters of Part III.