Numerical Computation

Size: px

Start display at page:

Download "Numerical Computation"

Shona Manning
6 years ago
Views:

1 An Introduction to Numerical Computation Downloaded from An Introduction to Numerical Computation

2 An Introduction to Numerical Computation Downloaded from This page intentionally left blank

3 An Introduction to Numerical Computation Downloaded from An Introduction to Numerical Computation Wen Shen Penn State University, USA World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI TOKYO

4 Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore USA office: 27 Warren Street, Suite , Hackensack, NJ UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE An Introduction to Numerical Computation Downloaded from British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. AN INTRODUCTION TO NUMERICAL COMPUTATION Copyright 2016 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. ISBN Printed in Singapore

5 An Introduction to Numerical Computation Downloaded from

6 An Introduction to Numerical Computation Downloaded from This page intentionally left blank

7 Preface An Introduction to Numerical Computation Downloaded from This is a set of lecture notes developed during a span of 10 years, for an introductory course on numerical computation which I have been teaching to senior undergraduate students at Penn State University. The course is cross-listed between the two departments of Mathematics and Computer Science. It is a standard 3-credit course, meeting 3 times per week, for a total of 15 weeks. These lecture notes contain precisely the material that can be covered in such a course, together with a few optional sections which may provide additional reading. The notes are written in a rather colloquial style, presenting the subject matter in the same form as it can be explained in a classroom. For instructors, this will minimize the amount of effort required to prepare their blackboard presentations. For students, these notes can also be used as a companion to the videos posted on my Youtube channel at youtube.com/wenshenpsu. These videos contain a complete set of live lectures for the course, which I recorded during the 2015 Spring semester. Shorter tutorials, lasting 5 to 15 minutes each and covering specific topics or examples, are also freely available on my Youtube channel. As prerequisites, students should have taken the standard calculus courses (in one and in several variables), an introductory course on matrices, and a course on computer programming. Having attended a sophomore-level course on differential equations is desirable, but not strictly required. After an introductory chapter on computer arithmetic, Chapters 2 4 of these notes cover polynomial approximations and their use for numerical integration. The following Chapters 5 7 are focused on the numerical solution of nonlinear equations and of linear systems. Data fitting, using least squares methods, is discussed in Chapter 8. The remainder of the book is concerned with the numerical solution of differential equations. Initial value problems for ODEs are covered in Chapter 9, while Chapter 10 describes some basic methods to solve two-point boundary value problems. Finally, the last chapter provides a brief introduction to finite difference schemes for partial differential equations. This is achieved by means of a few elementary examples, where the Laplace equation and the heat equations are solved on a rectangular grid, and the resulting systems of linear equations are discussed. For each class of problems, I decided to present only a few, basic computational vii

8 viii An Introduction to Numerical Computation An Introduction to Numerical Computation Downloaded from methods. Rather than surveying a large number of algorithms, I emphasize the key mathematical ideas that motivate these methods. In most cases, rigorous proofs are skipped, relying instead on graphs and drawings in order to build up intuition. Still, some simple proofs are fully worked out, introducing students to the elegance of mathematical reasoning. Implementation of the algorithms is an important part of the course. Although various program languages could be used, and many software packages are suitable for scientific computing, I here prefer to use Matlab. Tutorials on Matlab can be found online. Many Matlab examples and codes are included in this lecture notes, as a further help to students. A set of homework problems is included at the end of each chapter. These assignments typically include some simple calculations done with pencil and paper, together with larger computational projects, using Matlab. Lab projects cover a variety of applications, in connection with population dynamics, engineering, image reconstruction, mechanics, etc. They are designed to illustrate the wide range of applications and the power of numerical algorithms. Students in my class are required to complete every problem in the homework sets. In my experience, for many students this is a challenging course, requiring a combination of knowledge and skills of various kind. Students taking the class have different backgrounds, some of them majoring in mathematics, physics, engineering, computer science, etc. Some students are very fluent in computer coding, but may feel intimidated when confronted with an abstract proof. Others find it easy to understand mathematical ideas, but will need help with Matlab coding. It is this variety of backgrounds that makes it particularly hard to teach a course on numerical computation. Lectures must be designed so that every student receives the help he/she needs, and is motivated to learn new material. In the course of several years, these lecture notes have received very positive reviews from Penn State students. By publishing them as a book, and in conjunction with the Youtube videos, I hope a wider audience will also find them useful. I am grateful to my husband Alberto Bressan for carefully reading the draft, finding numerous errors, and making many suggestions for substantial improvement. Wen Shen Summer 2015 State College, Pennsylvania, USA

9 Contents An Introduction to Numerical Computation Downloaded from Preface 1. Computer Arithmetic Introduction Representation of Numbers in Different Bases Floating Point Representation Loss of Significance Review of Taylor Series and Taylor Theorem Numerical Differentiations and Finite Differences Homework Problems for Chapter Polynomial Interpolation Introduction Lagrange Interpolation Polynomials Newton s Divided Differences Errors in Polynomial Interpolation Properties of the Newton s Divided Differences (optional) Convergence of Polynomial Interpolation Homework Problems for Chapter Piecewise Polynomial Interpolation: Splines Introduction Linear Splines Quadratic Splines (optional) Natural Cubic Splines Homework Problems for Chapter vii 4. Numerical Integration Introduction Trapezoid Rule ix

10 x An Introduction to Numerical Computation 4.3 Simpson s Rule Recursive Trapezoid Rule Romberg Algorithm Adaptive Simpson s Quadrature Scheme Gaussian Quadrature Formulas Matlab Simulations A More Abstract Discussion (optional) Homework Problems for Chapter An Introduction to Numerical Computation Downloaded from 5. Numerical Solutions of Non-linear Equations Introduction Bisection Method Fixed Point Iterations Newton s Method Secant Method Systems of Non-linear Equations Homework Problems for Chapter Direct Methods for Systems of Linear Equations Introduction Naive Gaussian Elimination: Simplest Version Gaussian Elimination with Pivoting (optional) Difficulties with Naive Gaussian Elimination Gaussian Elimination with Partial Pivoting When Partial Pivoting is not Effective Gaussian Elimination with Scaled Partial Pivoting LU-Factorization (optional) Matlab Simulations Tridiagonal and Banded Systems Review of Linear Algebra Homework Problems for Chapter Fixed Point Iterative Solvers for Linear Systems General Introduction to Iterative Solvers Jacobi Iterations Gauss-Seidel Iterations SOR Iterations Matlab Simulations Writing All Three Methods in Matrix-Vector Form Error Analysis and Convergence Homework Problems for Chapter

11 Contents xi An Introduction to Numerical Computation Downloaded from 8. The Method of Least Squares Problem Description Linear Regression Method of Least Squares with Quadratic Functions Method of Least Squares with General Functions (optional) General Linear Method of Least Squares Matlab Simulations for Linear Method of Leasts Squares Non-linear Methods of Least Squares Method of Least Squares for Continuous Functions Homework Problems for Chapter Numerical Solutions of ODEs Introduction Taylor Series Methods for ODEs Runge-Kutta Methods An Adaptive Runge-Kutta-Fehlberg Method Multi-Step Methods A Case Study for a Scalar ODE, Solved in Matlab (optional) Euler s Method Heun s Method RK4 Method RKF5 Method Comparison Numerical Solution of Systems of First Order ODEs Higher Order Equations and Systems A Case Study for a System of ODEs by Various Methods (optional) Stiff Systems Homework Problems for Chapter Two-Point Boundary Value Problems Introduction Shooting Method Linear Shooting Some Extensions of Linear Shooting Non-linear Shooting Finite Difference Method Homework Problems for Chapter FDM for Partial Differential Equations Laplace Equation in 2D: Finite Difference Methods Some Extensions of the Laplace Equation Heat Equation in 1D

12 An Introduction to Numerical Computation Downloaded from xii An Introduction to Numerical Computation Forward-Euler Scheme for the Heat Equation Backward Euler Scheme for the Heat Equation Crank-Nicolson Scheme for the Heat Equation The θ-scheme for the Heat Equation Other Forms of the Heat Equation Homework Set for Chapter Bibliography 251 Index 253

Lecture Notes on Mathematical Olympiad Courses

Lecture Notes on Mathematical Olympiad Courses For Junior Section Vol. 2 Mathematical Olympiad Series ISSN: 1793-8570 Series Editors: Lee Peng Yee (Nanyang Technological University, Singapore) Xiong Bin