As an organizational note: none of us have enough hours in a day, and you probably won’t read most of the references below. In order to help sort out the ones I most recommend, I split each subsection into a main recommendation and other recommendations, and deliberately stick to one main recommendation.
This page is a living document. If there are good sources you think I’ve missed, please do submit a pull request against the main repository.
There is no required class for the text, and I will teach primarily from my own notes and slide decks, as well as from papers. Nonetheless, it’s good to have a reference or two.
- Structured Parallel Programming (McCool, Reinders, Robison)
- Programming Massively Parallel Processors (Kirk and Hwu)
- Performance Optimization of Numerically Intensive Codes (Goedecker)
- Principles of Parallel Programming (Lin and Snyder)
- Introduction to Parallel Programming (Pacheco)
- Parallel Programming (Wilkinson and Allen)
Related course materials
The closest course to CS 5220 (apart from previous instances of the class) is Jim Demmel’s Applications of Computers (Berkeley CS 267), available here in online form.
- Alan Edelman at MIT teaches a different parallel programming course, with the most recent iterations based on Julia. Materials are available here on MIT OpenCourseWare].
- Georgia Tech has two HPC courses; CSE 6220 and CSE 6230. Without exploring in detail, I think we’re closer to the latter.
- Randy Leveque at UW has also been experimenting with a flipped classroom for their intro HPC class, AM583. You may also be interested in the MOOC version.
Related resources from Intel
Intel has supported our course through a generous hardware donation. Part of the point is to develop new curricular content on parallel programming. They also have lots of existing curricular content and tutorials and references on Intel parallel technology, and these are worth a browse.
- Intel code modernization site
- Intel MIC developer site
- Video lectures on parallel computing
- Parallel programming curriculum initiative
- Parallel programming curriculum content
Recommended software skills background
Though I expect students taking the class to have some programming proficiency, that doesn’t necessarily mean everyone will have seen a Unix command line or a version control system. I mostly won’t spend class time on this, so you should take some time to brush up on your own.
Software Carpentry has good lessons on Unix shell
and Git. If you’re unfamiliar with
make, the lesson on Automation
and Make may also be useful (though less immediately critical).
- Cornell’s Center for Advanced Computing (CAC) offers a number of online training modules, including a good Linux introduction.
- GitHub has links to several good resources for learning about Git and GitHub. I was particularly amused by the interactive Try Git tutorial.
- Atlassian (makers of BitBucket) have a good [set of Git tutorials as well][bitbucket-turorials].
- ProGit is a freely available online book that covers the basics and much more.
As an aside: BitBucket is probably the primary competitor to Git. I recommend getting accounts on both GitHub and BitBucket; the latter provides unlimited private repositories for educational users, which is helpful for managing proposals, paper drafts, and other projects that you might want to keep initially private.
You will need to have prior programming experience in some C family language (Java, C, C++, C#), but that doesn’t mean you will have done any C programming before. We’re going to use C as the lingua franca for most of the class, so it’s a good idea to do some reading if you don’t know the language already.
[The C Programming Language (Kernighan and Ritchie)][kr-book] is the bible of C programming. I like it both as a reference to the language and an example of how to write an effective manual.
- The C intro from CAC is not a bad place to get started learning about C. In particular, check out their references page.
- Materials from an MIT course on Practical Programming in C also look pretty good.
- Cornell offers courses in C (CS 2022) and C++ (CS 2024).
Specifically, you will want to look at the lessons on the Unix shell and Git early on, and the lesson on Automation and Make fairly soon. For later in the class, you may want the lesson on Programming with Python.
Knowledge of Python is not strictly required for this class, but we will use Python in some of our work. We teach a Python course at Cornell (CS 1110) as our introductory course, but you are not the target audience for that class.
- The Software Carpentry lessons include a good Python lesson that focuses on computational science uses.
- The official Python documentation is not a bad place to start; see particularly the tutorial if you’re new to the language.
- The SciPy lecture notes are a nice intro to the SciPy ecosystem.
- Dive Into Python is a nice introduction to Python for experienced programmers coming from other languages.
- My personal favorite Python text is probably Python Essential Reference by David Beazley.
- I believe CS 1110 at Cornell has used Think Python and Thinking like a Computer Scientist as texts in the recent past.