Reading

I have never believed it possible to have too many books. You should not think of these as required reading, but rather as references that you might find helpful along the way. All the texts linked on this page are either open or can be accessed via the Cornell library, though you may need to use Passkey. We note abbreviations for the books where we actually assign readings.

If you want to know my take on much of the background for this class, I recommend the notes from my own course. I also have some background notes that might be summarized as “things I might’ve assumed you knew” (correctly or incorrectly).

I have taught variants of this course before, and you might find some of them useful as well.

General Numerics

If you want to refresh your general numerical analysis chops and have fun doing it, I recommend the Afternotes books by Pete Stewart. If you would like a more standard text that covers most of the background relevant to this class, you may like Heath’s book (expanded for the “SIAM Classics” edition). I was involved in a book on many of the same topics, together with Jonathan Goodman at NYU. O’Leary’s book on Scientific Computing with Case Studies is probably the closest of the lot to the topics of this course, with particularly relevant case studies. And Higham’s Accuracy and Stability of Numerical Methods is a magesterial treatment of all manner of error analysis (highly recommended, but perhaps not as a starting point).

Numerical Linear Algebra

I learned numerical linear algebra from Demmel’s book, and still tend to go to it as a reference when I think about how to teach. Trefethen and Bau is another popular take, created from when Trefethen taught at Cornell CS. Golub and Van Loan’s book on Matrix Computations ought to be on your shelf if you decide to do this stuff professionally, but I also like the depth of coverage in Stewart’s Matrix Algorithms (in two volumes). And Elden’s Matrix Methods in Data Mining and Pattern Recognition is one of the closest books I’ve found to the spirit of this course (or at least part of it).

Numerical Optimization

My go-to book on numerical optimization is Nocedal and Wright, with the book by Gill, Murray, and Wright as a close second (the two Wrights are unrelated). For the particular case of convex optimization, the standard reference is Boyd and Vandeberghe. And given how much of data fitting revolves around linear and nonlinear least squares problems, we also mention an old favorite by Bjorck.

Machine Learning and Statistics

This class is primarily about numerical methods, but the application (to tasks in statistics, data science, and machine learning) is important to the shape of the methods. My favorite book for background in this direction is Hastie, Tribshirani, and Friedman, but the first book I picked up (and one I still think is good) was Bishop. And while you may decide not to read the entirety of Wasserman’s book, I highly recommend at least reading the preface, and specifically the “statistics/data mining dictionary”.

Math Background

If you want a quick refresher of “math I thought you knew” and prefer something beyond my own notes, Garrett Thomas’s notes on “Mathematics for Machine Learning” are a good start. If you want much, much more math for ML (and CS beyond ML), the book(?) by Gallier and Quaintance will keep you busy for some time.