**Course overview:** Introduction to computational mathematics covering topics in (numerical) linear algebra, statistics, and optimization. Topics included are those of particular relevance to upper-division computer science courses in machine learning, numerical analysis, graphics, vision, robotics, and more. An emphasis is placed both on understanding core mathematical concepts and introducing associated computational methodologies.

**Instructor:** Anil Damle

**Contact:** damle@cornell.edu

**Office hours:** Mondays from 2:45 to 3:45 pm and Wednesdays from 2 to 3 pm in Gates 423

**TA:** Ian Delbridge

**Contact:** iad35@cornell.edu

**Office hours:** Tuesdays 10 to 11 am in Rhodes 412 and Thursdays 12 to 1 pm in Rhodes 406

**TA:** Anmol Kabra

**Contact:** ak2426@cornell.edu

**Office hours:** Mondays 10 to 11 am in Rhodes 412

**Lectures:** Monday, Wednesday, and Friday from 11:15 AM till 12:05 PM in Gates Hall G01

**Course websites:** Homework, projects, and exams will be turned in using the course management system (CMS). There will also be a course discussion forum run via Piazza.

- This webpage is currently under construction and some specifics may change between now and the first day of class
- August 30 — first day of class

Your grade in this course is comprised of three components: homeworks, exams, and a final project. Please also read through the given references in concert with the lectures.

**Homeworks:**There will be a number of homework assignments throughout the course, typically made available roughly one to two weeks before the due date. They will include a mix of mathematical questions and basic implementation of algorithms. Any required implementation may be done in MATLAB, Julia, or Python. Homeworks should be typeset (or, for the theoretical questions, very neatly handwritten and scanned) and submitted along with any associated code via the CMS.

**The project:**The goal of this course is to provide you with the mathematical and computational fundamentals that underpin a broad range of applications. Therefore, in lieu of a final exam this course will have an open ended final project. In particular, leveraging what you have learned about linear algebra, statistics, and optimization you will tackle a problem of interest to you (you may work individually or in a group and the maximum group size is TBD). Several example projects spanning a range of applications will be provided, though you may also propose your own. More details about the project will be provided within the first several weeks of class including specific requirements and due dates for proposals, check-ins, etc. Per university policy, the final project will be due

**December 20, 2019 at 11:30 AM**.**Exams:**There will be two short prelim exams for this class focused on the more theoretical material discussed in the first two thirds of the course. The specific scope of each exam will be provided closer to the exam date along with relevant practical information.

- First prelim exam: October 17, 2019 at 7:30 PM in Olin Hall 255 [Example problems] [Solutions to example problems]
- Second prelim exam: November 14, 2019 at 7:30 PM in Kimball Hall B11 [Example problems] [Solutions to example problems]

Your final grade in the course will be computed based on the homework assignments, project, exams, and course participation

- Participation: 5%
- Homework: 35%
- Project: 30%
- Prelim exams: 30% (15% each)

You are encouraged to actively participate in class. This can take the form of asking questions in class, responding to questions to the class, and actively asking/answering questions on the online discussion board. We will also be soliciting feedback mid-semester to hopefully improve the course.

You may discuss the homework and projects freely with other students, but please refrain from looking at code or writeups by others. You must ultimately implement your own code and write up your own solution.

Except for the final project report, all work is due at 11:59 pm on the due date. Homework and projects should be submitted via the CMS. For each assignment you are allowed up to two "slip days". However, over the course of the semester you may only use a total of eight slip days. You may not use slip days for the final project report.

Grades will be posted to the CMS, and regrade requests must be submitted within one week.

MATH 2210 or MATH 2940 or equivalent; pre- or co- requisite: one programming class, and some familiarity with probability and statistics

The Cornell Code of Academic Integrity applies to this course.

In compliance with the Cornell University policy and equal access laws, I am available to discuss appropriate academic accommodations that may be required for student with disabilities. Requests for academic accommodations are to be made during the first three weeks of the semester, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.

The text by Gilbert Strang serves as the primary reference for this course, though I will populate this section of the website with a collection of other resources you may find useful throughout the course.

**Linear Algebra and Learning from Data by Strang**

This book serves as the primary reference for this course and is one of the few that covers the blend of linear algebra, statistics, and optimization that we will in this course. Sections are abbreviated as GS in the references below.**Linear Algebra and its applications by Strang**

This book provides an additional resource for much of the linear algebra we will cover in this course. One version of it is available digitally through Cornell's library.**Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares by Boyd and Vandenberghe**

This book provides an additional resource for much of the linear algebra we will cover in this course and is freely available online. [Book website]**Linear Algebra and its applications by Lay, Lay, and McDonald**

This book provides an additional resource for much of the linear algebra we will cover in this course and is the one used in many of hte linear algebra courses here at Cornell.**A First Course in Numerical Methods by Ascher and Greif**

This book is the primary course text for CS 4220, but may also serve useful for some specific parts of this course (in particular some of our discussion on optimization). [SIAM online]**A First Course in Probability by Ross**

This book provides good coverage of some of the basic probability questions we have discussed such as sampling of random variables. It is readily available through the Cornell Library**Introduction to Mathematical Statistics by Hogg, McKean, and Craig**

This book provides good coverage of some of the basic statistics questions we have discussed such as maximum likelihood estimation. It is readily available through the Cornell Library**An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani**

This book provides a good overview of some methods in statistical learning, some of which we will discuss. The book is available online through the books website and via the Cornell Library. [Book website]

**Lecture notes by Embree**

A set of lecture notes by Mark Embree for a course in numerical analysis he taught while at Rice. While the content of the notes does not completely align with this course, they still may be a useful and interesting resource. [online]**Linear algebra course by Strang**

Portions of this course will utilize your knowledge of linear algebra. If you feel you need additional preparation, or would like to revisit the topic, you may find Gilbert Strangs linear algebra course quite useful. [MIT Open Courseware]**Julia**

If you need resources for the Julia language, the documentation and various resources (under the learning link) are available on the Julia website. If consulting the documentation, please make sure the documentation you are consulting is for the version matching what you are running. [Julia webpage]**MATLAB**

One useful resource for MATLAB is the book "Insight through Computing: A MATLAB Introduction to Computational Science and Engineering" by Van Loan and Fan. It is available for free online through Cornell's SIAM subscription. [SIAM online] You may also find the online documentation helpful. [MATLAB documentation]**LaTeX**

Further information about LaTeX, including how to obtain/install it, may be found at the projects homepage. In addition, there is a WikiBook you may find useful if you are looking to learn LaTeX. Lastly, you may find the online platform Overleaf (loosely speaking, a Google Docs for LaTeX) useful and they have their own tutorial.

A tentative schedule follows, and includes the topics we will be covering, relevant reference material, and assignment information. It is quite possible the specific topics covered on a given day will change slightly. This is particularly true for the lectures in the latter part of the course, and this schedule will be updated as necessary.

Date | Topic | References | Notes/assignments |
---|---|---|---|

8/30 | Introduction | ||

9/4 | Notation and fundamentals | GS: 1.1 and 1.2 | |

9/6 | Four fundamental subspaces | GS: 1.3 | |

9/9 | Orthogonal matrices and subspaces | GS: 1.3 and 1.5 | |

9/11 | Norms | GS 1.11 | |

9/13 | The SVD | GS 1.8 | HW 1 due |

9/16 | The SVD | GS 1.8 | SVD notes |

9/18 | The SVD | GS 1.8 | SVD demo code |

9/20 | Projection matrices and Gram-Schmidt | See additional resources and references | Orthogonal projector notes |

9/23 | Eigenvalues and eigenvectors | See additional resources and references | HW 2 due on 9/24 |

9/25 | Eigenvalues and eigenvectors | See additional resources and references | |

9/27 | Power metho | See additional resources and references | |

9/30 | Sensitivity of linear systems | See additional resources and references | Condition number notes |

10/2 | Matrix factorizations and linear systems | See additional resources and references | |

10/4 | Least Squares | See additional resources and references | |

10/7 | Randomized methods | ||

10/9 | Randomized methods | HW 3 due on 10/10 | |

10/11 | Floating point | ||

10/16 | Prelim review | ||

10/17 | Prelim exam at 7:30 PM | Prelim exam at 7:30 PM | |

10/18 | Floating point | ||

10/21 | Intro to probability | GS 5.1-5.3 | |

10/23 | Intro to probability | GS 5.1-5.3 | |

10/25 | Tail bounds | Ross | |

10/28 | Limit theorems and multivariate Gaussian | GS 5.5 | |

10/30 | Sampling | Ross 10.2 | |

11/1 | Sampling | Ross 10.2 | |

11/4 | Estimation | GS 5.1-5.3 | Homework 4 due |

11/6 | Estimation | GS 5.1-5.3 | |

11/8 | MLE | GS 5.5 | |

11/11 | PCA | Intro to Statistical Learning 10 | Homework 5 due |

11/13 | PCA | Intro to Statistical Learning 10 | |

11/14 | Prelim exam at 7:30 PM | Prelim exam at 7:30 PM | |

11/15 | Least squares with noise | Intro to Mathematical Statistics 12.3 | |

11/18 | What it means to solve an optimization problem | Ascher and Greif 3.5 and 9.2, and GS 6.1 and 6.4 | |

11/20 | Search direction methods | Ascher and Greif 3.5 and 9.2, and GS 6.1 and 6.4 | |

11/22 | Search direction methods | Ascher and Greif 3.5 and 9.2, and GS 6.1 and 6.4 | |

11/25 | Optimization demo | demo code | |

12/2 | No class, snow day | ||

12/4 | Lagrange Multipliers | ||

12/6 | Stochastic Gradient Descent | ||

12/9 | Flex | Homework 6 due | |

12/20 | Project due at 11:30 AM | Project due at 11:30 AM |