CS 5431 students are expected to participate in a team project to build a software system that has non-trivial security functionality. The goal of the project is for you to learn how to secure code that you write (in cruder terms: you need to eat your own dog food), and to learn something about software engineering along the way. Giving you pre-written code would defeat both of these goals, which is why we ask you to start from scratch in building your system. That does mean that you'll spend time writing code that is not directly related to security functionality. That's okay: this practicum sits at the intersection of security and software engineering.
This project is not a toy assignment. You'll be building a substantial software system all of your own design. You'll need to make many decisions and solve many problems along the way. It's a demanding job and—if approached with the right attitude—highly rewarding.
Contents:
Essential Functionality. This semester, you will build a password manager. Humans will use this application to manage their credentials for other computing systems. The overriding security concern in this project is that you should establish sufficient assurance in the security of your system that you would be willing to store your own online banking credentials in a public deployment of your system.
The required features of the password manager are as follows:
The second feature suggests that your implementation must involve a client and a server. The client is used by humans to interact with the system. The server is responsible for storing credentials. Although the client runs on a local machine, the server runs remotely, "in the cloud" (though you need not actually implement it as a cloud service).
Beyond those details, we impose no further requirements about the functionality of your system. Rather, we encourage you to be creative about creating a password manager that would be truly useful to humans who are faced with the contemporary challenge of managing credentials.
Note that fancy graphical user interfaces (GUIs) are not required, though nearly all teams build them anyway. Warning 1: a common mistake on this project is to spend too much time on the GUI and not enough time on the essential functionality and security elements. Warning 2: usability is an important concern in security and, indeed in all software. Concentrate more on usability than on aesthetics.
Essential Security Elements. Since you have leeway to establish your own functional requirements for the password manager, the security concerns for your system might differ from those of other teams. To ensure suitability as a course project, we therefore stipulate that projects are acceptable only if a majority (that is, at least 3 out of 5) of the following security elements are necessary for the system to fulfill its mission:
Availability is not a required element of the project. Many techniques for achieving availability are properly studied as part of a distributed systems or networks course.
Implementation. Select a programming environment, programming language, and target platforms that make sense for your project. Our evaluation of what you build will be based, in part, on your choice of tools (and our ability to understand your source code, so don't pick anything too obscure). Java and C# are safe choices.
If you elect to program in C or C++ then we will be interested in how you ensure that buffer overflows and other vulnerabilities that frequent C programs have been eliminated. There are tools for analyzing C and C++ source code for vulnerabilities, but running these tools (if you can get access to them) is a non-trivial undertaking.
If you elect to program in Java then we will encourage you to use the freely available security (and bug) analysis tool FindBugs. It would be best to integrate that tool into your development from the very beginning.
When building a system in industry, it's generally a good idea to extend existing components rather than build your own. For example, there are some third-party systems and tools available for securing web services. But using such tools in this course would preclude activities the project is supposed to cover. This is because, when you use a third-party tool, you must (i) accept somebody else's choices about what is useful security functionality and (ii) accept somebody else's assurance argument. We therefore adopt the following guiding principle:
However, we routinely make exceptions to this principle. Here are some standard exceptions:
Of course, the above list is incomplete, and in some cases it may make sense to incorporate other third-party code into your project. To do this, (i) you must have the instructor's prior approval, (ii) the license of that code must be amenable, and (iii) your project documents and presentations must clearly acknowledge the source of that code.
If you have questions about whether you can use third-party code that isn't listed above, then ask, rather than assuming one way or the other.
Public demonstration. Your system should be designed for demonstration to your classmates throughout the semester at each milestone, and possibly for public demonstration at the end of the semester. Our hope is that you will take the project as an opportunity to seriously impress your peers and potentially your future employers.
Why a team project? Because...
And most importantly...
Team formation. You are responsible for forming your own team. You can use the team formation features of Piazza to help with this.
Team size. Your team will start with three or four members, at your choice. Sometimes a student will drop, leaving an "orphaned" team member. We'll handle this on a case-by-case basis.
Peer reviews. As part of each milestone, you will submit peer reviews of your teammates. These will help the course staff to fairly assess the contribution of each team member to the team.
Everyone is responsible. All members of your team are ultimately responsible for understanding, hence being able to explain and defend, all security aspects of the system you build.
Your project grade will be calculated as follows:
Note how this grading scheme weights most of your grade toward your final system, and puts relatively little weight on the middle milestones (2 and 3). We've designed the scheme this way to encourage you to keep working without penalizing you for "going back to the drawing board." After all, in software development, things don't always work out as planned and you have to back up or start over.
Throughout the project, your grades will be influenced by the quality and non-artificiality of your project. Each milestone grade will also depend in part on whether you are making sufficient progress toward your final system.
Milestones. Each milestone will be assigned a letter grade. All team members will receive the same grade at each milestone, except for the final milestone.
Final system.
For the final milestone overall project grade,
each team member will receive their own (possibly different) grade. That
grade will be determined in part by the quality of your completed project,
your peer reviews, your demonstrations, and your individual ability to answer
questions about the security aspects of your project.
Challenge factor. For those students who truly seek an extra challenge, we include this factor as part of the grade. Most projects receive a challenge factor of around 0–3%. However, by making your project especially original or difficult, or by providing that extra "wow" factor, you can increase the challenge factor we assign to your project at the end of the semester. One way to increase your challenge factor is to learn and implement a cryptographic protocol from the research literature. Another way to increase your challenge factor is to build a system that is new and exciting. New means that your system should have some aspect that is novel, rather than just replicating some existing system. Exciting means that your system should have some aspect that provokes a response of "hey, that's cool!". In the ideal case, the exciting aspect of your software is also new. In that case, you might become rich and famous. :)
MEng students may use this project as the basis for the required MEng project. If this is your intention, then your team members must all be MEng students who are all electing to use the project in this manner. Your team will then be expected to satisfy the usual project requirements and also will be expected to:
You may start working on these additional tasks at any point during the semester. They must be completed before the end of the semester. And all other project milestones must be completed according to same schedule used by teams that are not exercising the MEng option.
An additional benefit of this project is that our team members became best friends. —anonymous student in fall 2011.
Your charter defines the members of your team and proposes the software system you intend to build. Your charter should include the following information:
The proposal section of your charter should be about one page long, which is considerably longer than the very brief system sketch provided above. In this section:
Submission. Form a group on CMS. As the group, submit a PDF to the Milestone 0 assignment.
Sample Charters. Some examples of past project charters will be posted later.
Prepare a Requirements Document that contains the six sections detailed below. This will be a living document: you will continue to update it in the remaining milestones of the project. It is also a critical document, because your final system will be evaluated, in part, against your requirements. You should think of the Requirements Document as a contract between your team and the course staff. You are responsible for fulfilling that contract.
1. Personnel. Identify your team members, including names, netids, and emails.
2. System Purpose. In a couple of paragraphs, explain the purpose of your system. The basis for this explanation should be the one-page system summary you wrote for Milestone 0. Update that summary as necessary, based on the functional requirements and security goals you develop in the current milestone, as well as any comments made by the course staff on your Milestone 0 submission.
3. Functional Requirements. Invent a list of functional requirements for the system you are building. For the purposes of this project, a functional requirement comprises the following data:
You might find it helpful to organize your requirements in a spreadsheet, so that you can sort them in various ways.
Here is an (incomplete!) example of four functional requirements, based on a grade management system for academic courses:
User type | Assets | Importance | User story |
---|---|---|---|
professor | assignment | M | As a professor, I can create a new assignment by specifying its name, number of possible points, and due date. |
student | submission | S | As a student, I can submit a file as a solution to an assignment. |
grader | grade | M | As a grader, I can assign a number of points as a grade to a student for an assignment. |
student | grade | S | As a student, I can view my grades for all assignments. |
Good user stories satisfy the following "INVEST" criteria:
The above user story examples satisfy the INVEST criteria reasonably well. But here's an example that doesn't satisfy the criteria:
U. type | Assets | Import. | User story |
---|---|---|---|
professor | survey | C | As a professor, I can conduct online surveys of students, so that I don't have to print and collect paper surveys. |
This user story doesn't specify what a survey is, how students will take it, how results will be tabulated and presented, etc. It should be broken down into smaller stories. And this is likely how you will proceed with inventing user stories: write stories that are too "big" at first, then refine them.
How many requirements should we invent? It's impossible to give a general answer to this question: teams write at different levels of detail, have differently scoped projects, and build systems with different tradeoffs between feature set size and implementation difficulty.
4. Threat Analysis. Identify threats of concern to your system. Concentrate primarily on malicious, human threats. What kinds of attackers will your system defend against? What are their motivations, resources, and capabilities? Don't just list vague, generic threats. Make them specific to your system and its functionality. If there are any non-threats, you should identify them, too. For example, you may wish to assume that some system components execute on hardware that is located in a physically secured machine room reachable only by trustworthy system operators.
5. Security Goals. Identify the assets and stakeholders involved with your system. This step should be easy, because you already identified assets and users for each functional requirement. For each asset, identify what its value is to stakeholders.
Perform a harm analysis on assets. Use the template "performing action on/to/with asset could cause harm." (Although you are encouraged to rewrite statements made with that template into more natural English.) Be as thorough and creative here as possible; this is the step at which you're most likely to overlook something that's important and relevant to security.
Transform the harms you've identified into security goals, using the template "the system shall prevent action on/to/with asset." Label each goal as being exactly one of confidentiality, integrity, or availability. Examine the feasibility of each goal in light of your threat analysis. If necessary, relax goals so that it is feasible to achieve them.
You must document all of these steps, showing us your asset analysis, harm analysis, feasibility analysis, and security goals.
Security Requirements. Actually, you should not produce security requirements (as they are defined in the lecture notes) as part of this milestone. You'll do that in milestones 2 through 4.
6. Essential Security Elements Document why each of the essential security elements is indeed essential to your system. If you can't construct a persuasive argument for an element E, you need to revisit your system purpose and invent new features that will cause E to be essential. Recall, however, that you need to involve only a majority of the elements.
Submission. As a group, submit a single PDF to the Milestone 1 assignment on CMS.
Evaluation. We will evaluate your Requirements Document against the following criteria:
Sample Requirements Documents. We will later post a couple sample requirements document from past semesters. These should not be regarded as perfect, but they do convey a good idea of what we are asking you to write.
All the remaining milestones for the project involve the same essential work. You will engage in a sprint, which is a fixed length of time during which you build a "shippable" increment of your system. Your sprints will last three weeks. At the end of each sprint, you will ship the latest increment of your system to the course staff and demo it for us. You have three sprint cycles—named alpha, beta, and final—to implement your project.
Read the project sprint overview to find out more.
When to implement what. You are empowered to decide what features you will deliver as part of each sprint. But we do require that you deliver "gold standard" security functionality according to the following schedule:
Teams that do not deliver security functionality according to this schedule will receive milestone grades that reflect unsatisfactory progress. Note that the schedule assumes that all elements are relevant to your team's system; in practice, some might not be.
Source Control. You are required to keep your project in a source control system. There are many free possibilities [GitHub, Bitbucket, SourceForge, etc.] that provide code hosting, bug tracking, and wikis. Under no circumstances should you resort to emailing your entire source code between team members.
Demo. During class, you will demo your alpha deliverable on the day that it is due. Bring laptops that have your system installed and running. The course staff and the other project teams will use your system and report on how well it appears to be working and achieving its security goals. Attendance of all team members is required at your team's demo. Demos typically involve some or all of the following events, though there is necessarily some flexibility involved:
Submission.
Bundle those files up into a zip file. Name it alpha.zip. As a group, submit that file to the Milestone 2 assignment.
Evaluation. Your demo will be evaluated on how well it demonstrates the functionality you are delivering for this milestone (i.e., the system backlog items that you have marked as completed), particularly security functionality. You should demo only completed (= working and tested) functionality. We will be especially unimpressed by any bugs we see during your demo, if those bugs relate to functionality that you claim to have completed.
The design and implementation of your system, including your code, will be evaluated on how well it exemplifies the various principles of security. Readability, simplicity, documentation, and testing matter. We'll also, of course, be looking for vulnerabilities.
Advice. In the past, students have sometimes come to office hours asking, "I'm thinking of implementing X. Is that okay?" By okay, they mean, "will that cause me to lose points because there's some insecurity lurking in it?" That's not a profitable question to ask. If the course staff answered such questions, it would be robbing you of the intellectual exercise you must go through to become an expert in security. In this course, figuring out whether something is secure is first your job. You must develop enough confidence in your own solutions that you're willing to claim they're secure. After all, that's what you'll be doing after this class in the real world. So a much better question to ask is, "I'm thinking of implementing X. Here's why I think X is secure. [Your argument.] But I'm not sure about [some small piece of your argument]. How could I become more sure?"
Note, however, that even if the the course staff with whom you are talking doesn't initially think of a possible attack, they (or another staff member or another 5431 student) might later discover one. Regardless, any insecurity in a project is grounds for losing points. Although this might seem unfair, it's life: in the real world, when you deploy a system that has unknown vulnerabilities, and an attacker later discovers and exploits them, you can't use "I thought it was secure" as an excuse to undo any harm done by the attacker.
Complete another sprint and ship the latest increment of your system to the course staff. Your beta milestone should exhibit new features and new security functionality that were not present in your alpha demo.
Demo, Submission, and Evaluation. Details about these remain the same as in Milestone 2. As a group, submit to the Milestone 3 assignment.
Complete a final sprint and ship your completed system to the course staff. All your essential security elements, as well as the core functionality of your system, must be completed.
Submission. Details about submission remain the same as in Milestone 2. Note that you might have uncompleted system backlog items for your final release; that's a fact of life when it comes to building software systems. As a group, submit to the Milestone 4 assignment.
Evaluation. The design and implementation of your system, including your code, will be evaluated on how well it exemplifies the various principles of security, the extent to which your system fulfills its purpose, realizes its security goals against threats of concern, and involves the Essential Security Elements. Readability, simplicity, documentation, and testing matter. We'll also, of course, be looking for vulnerabilities. The originality, difficulty, and non-artificiality of your project will also influence the evaluation.
Your team will present your completed system to the course staff at a date near the final 5431 class meeting. Your will sign up for its own presentation slot; other teams will not be present. Rehearse your presentation well, because quality counts and time is tight. The presentation may be given by one member of your team, or you may "tag team." Plan to use your own laptops to present.
Your presentation will proceed in three phases:
Phase 1. You will give an uninterrupted talk of no more than 12 minutes using slides. Slides should neither be too heavy on text nor too light on content. Use figures instead of text where possible. Your talk must address the following topics:
Hints: (i) Structure your talk as six sections that parallel the six topics above. (ii) Script the talk, so that you know exactly what you plan to say. But bear in mind that no one enjoys a talk in which it sounds like the speaker is simply reading from a script. (iii) It's better to tell us up front the shortcomings of your system than for us to discover them through questioning.
Phase 2. You will demo your system. The demo should last no more than 8 minutes. The demo must illustrate live functionality of your system. Your audience will be especially unimpressed by any failures that are seen during the demo.
Hints: Enforce a feature freeze well before the demo, so that you can devote sufficient effort to finding and fixing faults. And enforce a code freeze the day before the demo, so that you don't accidentally introduce new faults at the last minute.
Phase 3. We will hold a question and answer session. The audience asks; you answer. Anything about your system is fair game, including requests for you to demo something new about your system or review some source code. Questions may be directed to any member of your team.
Hint: Get enough sleep the night before. It's really hard to answer questions in front of an audience when you haven't slept.
Submission. Submit your presentation slides to the Presentation assignment on CMS.
Evaluation. You will be evaluated on the content and quality of the presentation, your slides and delivery, whether the demo robustly works and illustrates your security functionality, and your performance during the Q&A session.
Complete this peer review form for each member of your group including yourself. This will help us understand the comments that others wrote about you, and whether you would consider those comments to be fair. Your comments will be treated as confidential and will not be shared with the other members of your group.
Among other things, the form asks you to assign each member a letter grade (A+ through F) in each of the following areas:
As a general rule of thumb, an A indicates "impressive", a B is "adequate", and a C indicates "many problems".
Submission. Submit a single document. Start each member's review on a new page. In each review, clearly identify both your name and the name of the person you are reviewing. (The form provided above already does both these things.) Submit to the Project Peer Review assignment on CMS.
Evaluation. You will be evaluated on whether you followed the above instructions. Especially, you must submit a full review for every member of your team, including yourself, using the supplied form.
Here is some sarcastic (and hopefully humorous) advice on how to fail your project, based on mistakes commonly seen year after year.