Skip to main content

Project

CS 5430 students are expected to participate in a team project to build a software system that has non-trivial security functionality. In our experience, this project is the best way for you to master the content of the course. Realize that the goal of the project is for you to learn how to secure code that you write (in cruder terms: you need to eat your own dog food), and to learn something about software engineering along the way. Giving you pre-written code would defeat both of these goals, which is why we ask you to start from scratch in building your system. That does mean that you'll spend time writing code that isn't directly related to security functionality. That's okay: remember, this course sits at the intersection of security and software engineering.

This project is not a toy assignment like you might be used to from other courses. You'll be building a substantial software system all of your own design. You'll need to make many decisions and solve many problems along the way. It's a demanding job and—if approached with the right attitude—highly rewarding.

Overview

Essential Functionality

This semester, you will build a password manager. Humans will use this application to manage their credentials for other computing systems. The overriding security concern in this project is that you should establish sufficient assurance in the security of your system that you would be willing to store your own online banking credentials in a public deployment of your system.

The required features of the password manager are as follows:

  • Humans use the password manager to create and store usernames and passwords for many other systems.
  • Users can access the password manager from many machines (their laptop, their desktop, maybe their phone, etc.).

The second feature suggests that your implementation must involve a client and a server. The client is used by humans to interact with the system. The server is responsible for storing credentials. Although the client runs on a local machine, the server runs remotely, "in the cloud" (though you need not actually implement it as a cloud service).

Beyond those details, we impose no further requirements about the functionality of your system. Rather, we encourage you to be creative about creating a password manager that would be truly useful to humans who are faced with the contemporary challenge of managing credentials.

Note that fancy graphical user interfaces (GUIs) are not required, though nearly all teams build them anyway. Warning 1: a common mistake on this project is to spend too much time on the GUI and not enough time on the essential functionality and security elements. Warning 2: usability is an important concern in security and, indeed in all software. Concentrate more on usability than on aesthetics.

Essential Security Elements

Since you have leeway to establish your own functional requirements for the password manager, the security concerns for your system might differ from those of other teams. To ensure suitability as a course project, we therefore stipulate that projects are acceptable only if a majority (that is, at least 3 out of 5) of the following security elements are necessary for the system to fulfill its mission:

  • Authorization. The system must enforce some non-trivial authorization policy to control some subset of its operation. Implement an access control mechanism that is appropriate and natural for your system's functionality and expected scale.
  • Authentication. The system must authenticate its clients or allow its clients to authenticate the system. Clients may include human users as well as programs executing on machines. You must implement reasonable means of authentication, which may include passwords, user registration, generation of secrets, distribution of keys, etc.
  • Audit. The system must provide infrastructure for audit or other means of establishing accountability for actions.
  • Confidentiality and Integrity. The system must involve information that resides in long-term storage or that is transmitted over a network. The system's mission must require that information to be kept secret and/or be protected from corruption.

Availability is not a required element of the project. Many techniques for achieving availability are properly studied as part of a distributed systems or networks course.

Implementation

You must use Java to implement your system. Java prevents buffer overflows and other vulnerabilities, and we will be using source-code analysis tools and security libraries that work only with Java. No exceptions will be made to this requirement.

When building a system in industry, it's generally a good idea to extend existing components rather than build your own. For example, there are some third-party systems and tools available for securing web services. But using such tools in this course would preclude activities the project is supposed to cover. This is because, when you use a third-party tool, you must (i) accept somebody else's choices about what is useful security functionality and (ii) accept somebody else's assurance argument. We therefore adopt the following guiding principle:

BYOS: Build your own system. All code in your system must be your own.

However, we routinely make exceptions to this principle. Here are some standard exceptions:

  • Java's standard libraries (i.e., those part of the distribution) may be used. This includes various cryptographic routines, which you shouldn't be writing yourself anyway.
  • The Bouncy Castle crypto library may be used.
  • GUI builders may be used.
  • Standard operating systems may be used. This includes the networking infrastructures and file systems native to those operating systems. It does not include web services frameworks (including browsers and web servers), which you are not permitted to use.
  • If you build a component that runs on a mobile platform (e.g., Android or iOS), the standard libraries and frameworks on that platform may be used.
  • Database management systems may be used. Warning: don't take that as a suggestion that you should use a DBMS. Projects often founder when DBMS's are involved.

Of course, the above list is incomplete, and in some cases it may make sense to incorporate other third-party code into your project. To do this, (i) you must have the instructor's prior approval, (ii) the license of that code must be amenable, and (iii) your project documents and presentations must clearly acknowledge the source of that code.

If you have questions about whether you can use third-party code that isn't listed above, then ask, rather than assuming one way or the other.

Public demonstration

Your system should be designed for demonstration to your classmates throughout the semester at each milestone, and possibly for public demonstration at the end of the semester. Our hope is that you will take the project as an opportunity to seriously impress your peers and potentially your future employers.

Teams

Why a team project? Because...

  • Working in a team offers you the powerful tool of discussing ideas with others.
  • Working in a team affords the opportunity for parallel development activities and specialized expertise.
  • Working in a team helps hone skills needed to be effective in the workplace, where teams are the norm.

And most importantly...

  • The best way to begin learning security is to try to secure something you've built yourself. One person alone usually can't build a sufficiently interesting system to secure.

Team formation

You are responsible for forming your own team. You can use the team formation features of Piazza to help with this.

Team size

Your team will start with three or four members, at your choice. Sometimes a student will drop, leaving an "orphaned" team member. We'll handle this on a case-by-case basis.

Peer reviews

As part of each milestone, you will submit peer reviews of your teammates. These will help the course staff to fairly assess the contribution of each team member to the team.

Everyone is responsible

All members of your team are ultimately responsible for understanding, hence being able to explain and defend, all security aspects of the system you build.

Grading

Your project grade will be calculated as follows:

  • Milestone 0 (Charter): 5%
  • Milestone 1 (Requirements): 20%
  • Milestone 2 (Alpha): 10%
  • Milestone 3 (Beta): 10%
  • Milestone 4 (Final system): 40%
  • Final peer review: 5%
  • Challenge factor: 10%

Note how this grading scheme weights most of your grade toward your final system, and puts relatively little weight on the middle milestones (2 and 3). We've designed the scheme this way to encourage you to keep working without penalizing you for "going back to the drawing board." After all, in software development, things don't always work out as planned and you have to back up or start over.

Throughout the project, your grades will be influenced by the quality and non-artificiality of your project. Each milestone grade will also depend in part on whether you are making sufficient progress toward your final system.

Milestones

Each milestone will be assigned a letter grade. All team members will receive the same grade at each milestone, except for the final milestone.

Final system

For the final milestone, each team member will receive their own (possibly different) grade. That grade will be determined in part by the quality of your completed project, your peer reviews, your demonstrations, and your individual ability to answer questions about the security aspects of your project.

Challenge factor

For those students who truly seek an extra challenge, we include this factor as part of the grade. Most projects receive a challenge factor of around 0%. However, by making your project especially original or difficult, or by providing that extra "wow" factor, you can increase the challenge factor we assign to your project at the end of the semester. One common way to get a substantial challenge factor is to eliminate any trust assumptions about a central server: trust must somehow be distributed over a set of principals. Trusted clients might also need to be eliminated.

Make no mistake: projects with a non-zero challenge factor are truly much more difficult to build. They typically require learning and implementing new cryptographic protocols from the research literature. In the last couple offerings of this course, only a couple projects managed to attain a challenge factor greater than 3%. Don't go for this option unless you're prepared to devote a large part of your life this semester to this project. If you want a good grade in the course, you're better off concentrating on building a very well-secured system that makes stronger trust assumptions.

Another way to increase your challenge factor is to build a system that is new and exciting. New means that your system should have some aspect that is novel, rather than just replicating some existing system. Exciting means that your system should have some aspect that provokes a response of "hey, that's cool!". In the ideal case, the exciting aspect of your software is also new. In that case, you might become rich and famous. :)

MEng Option

MEng students may use their CS 5430 project as the basis for the required MEng project. If this is your intention, then your team members must all be MEng students who are all electing to use the project in this manner. Your team will then be expected to satisfy the usual project requirements and also will be expected to:

  • Implement some additional functionality, as agreed upon between your group and the course staff.
  • Provide a written "project report" for the entire project. This report should document the problem solved, design alternatives, a rationale for design selected, the implementation, the testing strategy employed, and give instructions for both installation and use.

You may start working on these additional tasks at any point during the semester. They need not be completed before the end of the semester. But all other project milestones must be completed according to same schedule used by teams that are not exercising the MEng option.

Milestone 0: Charter

An additional benefit of this project is that our team members became best friends. —anonymous student in fall 2011.

Your charter defines the members of your team and proposes the software system you intend to build. Your charter should include the following information:

A working name for your system.
You are not committed to this name; it may evolve throughout the semester.
The members of your team and their contact information.
List the names and NetIDs of your team members.
Your plan for your weekly status meeting.
We strongly recommend that, as a team, you agree on a weekly meeting time and place (outside of class) that you can both attend to discuss the status of your project. Benefit from our experience: teams that actually do this seem to enjoy more success.
A proposal for your system.
Summarize the system you intend to build. You should regard this summary as a starting point; you will have the chance to elaborate in future milestones. Here, you should tell us what will be the most important functionality of your system.

The proposal section of your charter should be about one page long, which is considerably longer than the very brief system sketch we provided to you. In this section:

  • Provide a very short statement of the core vision or key idea for your system.
  • Provide a short (no more than six items) bulleted list of the key features of the system.
  • Provide a bulleted list of the essential security elements, and a one sentence description of how your system will incorporate each element.
  • Provide a narrative description of the system you intend to build. Go into enough detail in the body that, if your proposal were given to another team, and that team were never allowed to talk to you, they could still build more or less the system that you have in mind.

Formatting

Keep your proposal skimmable: someone should be able to look at the document quickly and get the basic idea, and quickly identify where they need to read for the details. In practice, this means:

  • Lots of subheadings and very short paragraphs.
  • Appropriate usage of font size and bold to draw reader attention.
  • Information early in the document should be short and punchy.
  • Complicated details should be left to the very end.

Submission

Form a group on CMS. As the group, submit a PDF to the Milestone 0 assignment.

Sample Charters

Some examples of past project charters will be posted later.

Milestone 1: Requirements

Prepare a Requirements Document that contains the information detailed below. This will be a living document: you will continue to update it in the remaining milestones of the project. It is also a critical document, because your final system will be evaluated, in part, against your requirements. You should think of the Requirements Document as a contract between your team and the course staff. You are responsible for fulfilling that contract.

Personnel

Identify your team members, including names, netids, and emails.

System Purpose

In a couple of paragraphs, explain the purpose of your system. The basis for this explanation should be the one-page system summary you wrote for Milestone 0. Update that summary as necessary, based on the functional requirements and security goals you develop in the current milestone, as well as any comments made by the course staff on your Milestone 0 submission.

Functional Requirements

Invent a list of functional requirements for the system you are building. For the purposes of this project, a functional requirement comprises the following data:

  • User type. The type of user involved in this requirement.
  • Assets. The critical information involved in this requirement.
  • Importance. The importance of this requirement to your system's purpose. Use the following "MoSCoW" scale to rank importance:
    • M. Must have.
    • S. Should have, if at all possible.
    • C. Could have, but not necessary.
    • W. Won't have now, but Would like to add later.
  • A user story. A brief description of a single kind of interaction a user can have with your system. User stories describe what your system will do, but not how. Here is a general template for a user story: "As a user, I can action, so that purpose." (See below for examples.) The purpose clause of a story might be obvious enough to omit that clause. Note: stories shouldn't be about developers or attackers.

You might find it helpful to organize your requirements in a spreadsheet, so that you can sort them in various ways.

Here is an (incomplete!) example of four functional requirements, based on a grade management system for academic courses:

U. typeAssetsImport.User story
professorassignmentM As a professor, I can create a new assignment by specifying its name, number of possible points, and due date.
studentsubmissionS As a student, I can submit a file as a solution to an assignment.
gradergradeM As a grader, I can assign a number of points as a grade to a student for an assignment.
studentgradeS As a student, I can view my grades for all assignments.

Good user stories satisfy the following "INVEST" criteria:

  • Independent. Stories shouldn't overlap, and ideally you should be able to implement them in any order—though some orders might make more sense than others.
  • Negotiable. Stories might (and probably will) need to change as your project evolves.
  • Valuable. Stories add value to your system for users. The general template above ensures that you think in terms of that value.
  • Estimable. Stories' importance can be estimated, hence ranked. And stories' implementation time can be estimated. If you can't estimate, you probably don't understand the story. Consider adding more detail or breaking it into smaller stories.
  • Small. Stories should be succinct, probably no more than one to four sentences.
  • Testable. Stories represent testable goals. If you can't decide whether you've met a goal—that is, if you can't write a test that a skilled outsider could follow to determine whether your system fulfills a story—then your story isn't clear enough.

The above user story examples satisfy the INVEST criteria reasonably well. But here's an example that doesn't satisfy the criteria:

U. typeAssetsImport.User story
professorsurveyC As a professor, I can conduct online surveys of students, so that I don't have to print and collect paper surveys.

This user story doesn't specify what a survey is, how students will take it, how results will be tabulated and presented, etc. It should be broken down into smaller stories. And this is likely how you will proceed with inventing user stories: write stories that are too "big" at first, then refine them.

How many requirements should we invent? It's impossible to give a general answer to this question: teams write at different levels of detail, have differently scoped projects, and build systems with different tradeoffs between feature set size and implementation difficulty.

Threat Analysis

Identify threats of concern to your system. Concentrate primarily on malicious, human threats. What kinds of attackers will your system defend against? What are their motivations, resources, and capabilities? Don't just list vague, generic threats. Make them specific to your system and its functionality.

In addition, if there are any non-threats, you should identify them, too. For example, you may wish to assume that some system components execute on hardware that is located in a physically secured machine room reachable only by trustworthy system operators.

Security Goals

Identify the assets and stakeholders involved with your system. This step should be easy, because you already identified assets and users for each functional requirement. For each asset, identify what its value is to stakeholders.

Perform a harm analysis on assets. Use the template "performing action on/to/with asset could cause harm." (Although you are encouraged to rewrite statements made with that template into more natural English.) Be as thorough and creative here as possible; this is the step at which you're most likely to overlook something that's important and relevant to security.

Transform the harms you've identified into security goals, using the template "the system shall prevent action on/to/with asset." Label each goal as being exactly one of confidentiality, integrity, or availability. Examine the feasibility of each goal in light of your threat analysis. If necessary, relax goals so that it is feasible to achieve them.

You must document all of these steps, showing us your asset analysis, harm analysis, feasibility analysis, and security goals.

Security Requirements

Actually, you should not produce security requirements (as they are defined in the lecture notes) as part of this milestone.

Essential Security Elements

Document why each of the essential security elements is indeed essential to your system. If you can't construct a persuasive argument for an element E, you need to revisit your system purpose and invent new features that will cause E to be essential. Recall, however, that you need to involve only a majority of the elements.

Submission

As a group, submit a single PDF to the Milestone 1 assignment on CMS.

Evaluation

We will evaluate your Requirements Document against the following criteria:

  • Whether your document is easy to understand, well organized, and clearly written. Correct English style and usage counts here, just as it does in the real world.
  • Whether the system purpose gives a clear summary of the system you are building.
  • Whether the functional requirements seem sensible given the purpose of the system.
  • Whether the functional requirements are clear and seem complete.
  • Whether the functional requirements adhere to the INVEST criteria.
  • Whether the threats of concern and their characteristics are described clearly.
  • Whether you have documented all the steps you went through in crafting your security goals, including your harm and feasibility analyses.
  • Whether you have correctly identified each security goal as being related to confidentiality, integrity, or availability.
  • Whether the security goals you identify are sensible for the system purpose, planned functionality, and threats of concern.
  • Whether your have successfully argued that a majority of security elements are involved in your system.

Sample Requirements Documents

We will post a couple sample requirements document from past semesters. These should not be regarded as perfect, but they do convey a good idea of what we are asking you to write.

Milestone 2: Alpha Release

All the remaining milestones for the project involve the same essential work. You will engage in a sprint, which is a fixed length of time during which you build a "shippable" increment of your system. Your sprints will last three weeks. At the end of each sprint, you will ship the latest increment of your system to the course staff and demo it for us. You have three sprint cycles—named alpha, beta, and final—to implement your project.

Read the project sprint overview to find out more.

When to implement what. You are empowered to decide what features you will deliver as part of each sprint. Nonetheless, we require that you deliver security functionality according to the following schedule:

  • Alpha: Audit; Cryptography for confidentiality and integrity
  • Beta: Authentication of humans
  • Final: Authorization

Teams that do not deliver security functionality according to this schedule will receive milestone grades that reflect unsatisfactory progress. Note that the schedule assumes that all elements are relevant to your team's system; in practice, some might not be.

Source Control

For your own good, you are required to keep your project in a source control system. There are many free possibilities [1, 2, 3, etc.] that provide code hosting, bug tracking, wikis, etc. Under no circumstances should you resort to emailing your entire source code between team members.

Demo

You will demo your alpha deliverable on the day that it is due. Bring laptops that have your system installed and running. The course staff will use your system and report on how well it appears to be working and achieving its security goals. Attendance of all team members is required at your team's demo.

Demos typically proceed as follows, though there is necessarily some flexibility involved:

  • You demo a few user stories that you've marked as completed.   Any bugs seen during that demo would be especially unimpressive.  The course staff might try weird inputs, strange user behavior, etc., in an attempt to find such bugs.
  • You exhibit source code for features you are delivering.  Code quality matters.
  • You discuss the design and implementation of any security functionality you are delivering.
  • You present a current FindBugs scan of your system.  High-ranked bugs, especially security bugs, would be of particular interest to the course staff.
  • The course staff asks you questions. Questions about the security of your system could be directed to any specific member of the group. All members should be able to explain and defend the security-relevant choices your group made.

Submission

  1. Submit a PDF containing your revised Requirements Document. Name that file requirements.pdf. The Functional Requirements section of your Requirements Document should now be renamed to the "System Backlog" section. You will continue to update it throughout the project. When you complete items, mark them completed and move them to a "Completed" part of the backlog. This section of your document is especially important, because we will use it to evaluate your progress. The rest of the Requirements Document should contain your personnel, system purpose, threat analysis, security goals, and essential security elements, all of which should continue to be updated throughout the project. You do not need to include the rest of the analysis from your Milestone 1 Requirements Document, in particular the asset, stakeholder, harm, or feasibility analyses—their purpose was to get you to security goals, which you now have.
  2. Create a new file, design.pdf, which describes the design of the security functionality of your system. This file should have five sections, one for each Essential Security Element. Fill in this file as appropriate for each milestone and for your system.
    • In the alpha milestone, for example, you should fill in the confidentiality, integrity, and audit sections, but leave the authentication and authorization sections blank, because you probably have not yet implemented that functionality. For this milestone, it would be appropriate to discuss issues such as how you generate and erase keys, how and where keys are stored, the exact cryptographic algorithms being used, sizes of keys and nonces, and assumptions about operating system functionality.
    • In the beta milestone, update the confidentiality, integrity, and audit sections as appropriate. You will add the design of authentication (of humans). Issues to discuss will include methods by which secrets (e.g., passwords) are chosen/generated, stored, updated, and erased.
    • In the final milestone, complete the document by updating all existing sections and finishing the authorization section. Issues to discuss will include the types of authorization policies being used (DAC vs. MAC), how they are implemented, how you have built a reference monitor, and how changes to policies are handled.
  3. Create another new file, assurance.pdf, which describes how and what you tested in the source code being submitted. Issues to discuss include how you tested individual methods, classes, and units; how you tested the integration of units, especially those written by different programmers; how you tested the features you are delivering; how you tested to ensure that security requirements are met; whether you did any pair programming or code reviews; how much code coverage your test suite achieves; and how often you run regression tests.
  4. Perform a FindBugs scan using the FindBugs GUI. For all bugs deemed Scary or Scariest, review each bug, classifying it and (if your classification is that the bug is not a bug or is harmless) making comments about the rationale for your classification. Save the result as a FindBugs analysis file named fbscan.xml, and submit that file.
  5. Submit a zip file, src.zip, containing the source of your system. (That is, .java source files, any .jar libraries you might be using, and whatever other source files are needed to build your system from scratch. Don't submit compiled .class files. And do not just zip up an Eclipse workspace.) Include a plain text file named "README" in root of your source detailing how to compile and execute your system on our own machines.
  6. And submit your sprint report along with your system. Name that file report.pdf.

Bundle those six files up into a zip file. Name it alpha.zip. As a group, submit that file to the Milestone 2 assignment.

Evaluation

Your demo will be evaluated on how well it demonstrates the functionality you are delivering for this milestone (i.e., the system backlog items that you have marked as completed), particularly security functionality. You should demo only completed (= working and tested) functionality. We will be especially unimpressed by any bugs we see during your demo, if those bugs relate to functionality that you claim to have completed.

The design and implementation of your system, including your code, will be evaluated on how well it exemplifies the various principles of security. Readability, simplicity, documentation, and testing matter. We'll also, of course, be looking for vulnerabilities.

Advice

In the past, students have sometimes come to office hours asking, "I'm thinking of implementing X. Is that okay?" By okay, they mean, "will that cause me to lose points because there's some insecurity lurking in it?"

That's not a profitable question to ask. If I answered such questions, it would be robbing you of the intellectual exercise you must go through to become an expert in security. In this course, figuring out whether something is secure is first your job. You must develop enough confidence in your own solutions that you're willing to claim they're secure. After all, that's what you'll be doing after this class in the real world.

So a much better question to ask is, "I'm thinking of implementing X. Here's why I think X is secure. [Your argument.] But I'm not sure about [some small piece of your argument]. How could I become more sure?"

Note, however, that even if the instructor doesn't initially think of a possible attack, he (or a TA or another student) might later discover one. Regardless, any insecurity in a project is grounds for losing points. Although this might seem unfair, it's life: in the real world, when you deploy a system that has unknown vulnerabilities, and an attacker later discovers and exploits them, you can't use "I thought it was secure" as an excuse to undo any harm done by the attacker.

Milestone 3: Beta Release

Complete another sprint and ship the latest increment of your system to the course staff. Your beta milestone should exhibit new features and new security functionality that were not present in your alpha demo.

Demo, Submission, and Evaluation

Details about these remain the same as in Milestone 2. As a group, submit to the Milestone 3 assignment.

Milestone 4: Final Release

Complete a final sprint and ship your completed system to the course staff. All your essential security elements, as well as the core functionality of your system, must be completed.

Submission

Details about submission remain the same as in Milestone 2. Note that you might have uncompleted system backlog items for your final release; that's a fact of life when it comes to building software systems. Include the slides for your final presentation as a file named slides.pdf in your submission archive. As a group, submit to the Milestone 4 assignment.

Evaluation

The design and implementation of your system, including your code, will be evaluated on how well it exemplifies the various principles of security, the extent to which your system fulfills its purpose, realizes its security goals against threats of concern, and involves the Essential Security Elements. Readability, simplicity, documentation, and testing matter. We'll also, of course, be looking for vulnerabilities. The originality, difficulty, and non-artificiality of your project will also influence the evaluation.

Milestone 4: Presentation and Demo

Your team will present your completed system to course staff during the final week (or two) of classes. You will sign up for individual presentation slots, some of which will occur during the regularly scheduled class time, and some of which will occur at other times.

Rehearse your presentation well, because quality counts and time is tight. The presentation may be given by one member of your team, or you may "tag team." Plan to use your own laptops to present.

Your presentation will proceed in three phases.

Phase 1. You will give an uninterrupted talk of no more than 12 minutes using slides. Slides should neither be too heavy on text nor too light on content. Use figures instead of text where possible. Your talk must address the following topics:

  1. Your team personnel.
  2. Your system's purpose.
  3. Your threat analysis.
  4. Your most important security goals.
  5. The design of the security of your system, including any trade-offs you made and any shortcomings you're aware of in your final system.
  6. How your project incorporates each of the essential security elements.

Hints: (i) Structure your talk as six sections that parallel the six topics above. (ii) Script the talk, so that you know exactly what you plan to say. But bear in mind that no one enjoys a talk in which it sounds like the speaker is simply reading from a script. (iii) It's better to tell us up front the shortcomings of your system than for us to discover them through questioning.

Phase 2. You will demo your system. The demo should last no more than 8 minutes. The demo must illustrate live functionality of your system. Your audience will be especially unimpressed by any failures that are seen during the demo.

  • If your system is designed to operate over a network, your demo must involve physically distinct machines that connect over a network. We will have two projectors available, which should be helpful for demoing on two laptops simultaneously.
  • If your system manages some kind of data repository (e.g., a relational database), have that repository pre-initialized with meaningful data, so that you don't have to spend demo time on just creating entries in the repository.

Hints: Enforce a feature freeze well before the demo, so that you can devote sufficient effort to finding and fixing faults. And enforce a code freeze the day before the demo, so that you don't accidentally introduce new faults at the last minute.

Phase 3. We will hold a question and answer session. The audience asks; you answer. Anything about your system is fair game, including requests for you to demo something new about your system or review some source code. Questions may be directed to any member of your team.

Hint: Get enough sleep the night before. It's really hard to answer questions in front of an audience when you haven't slept.

Evaluation

You will be evaluated on the content and quality of the presentation, your slides and delivery, whether the demo robustly works and illustrates your security functionality, and your performance during the Q&A session.

Milestone 4: Peer Review

Complete this peer review form for each member of your group including yourself. This will help us understand the comments that others wrote about you, and whether you would consider those comments to be fair. Your comments will be treated as confidential and will not be shared with the other members of your group.

Among other things, the form asks you to assign each member a letter grade (A+ through F) in each of the following areas:

Participation
This member consistently and punctually showed up to meetings, attended milestone presentations and work sessions, and contributed to discussions within the group.
Focus
This member stayed oriented on their assigned tasks. They consistently met deadlines. They took initiative to work on the project and get things done, even without being asked.
Quantity of Work
This member contributed a fair share of work. They carried their own weight.
Quality of Work
This member produced good work. They became knowledgable about their assigned duties and did not have to be reassigned because of poor work.
Attitude
This member respected other ideas and opinions within the group. They contributed to a positive and motivated atmosphere. It was a pleasure to work with them.
Overall Grade
Based on all of the items that you have evaluated above, assign an overall grade to this member.

As a general rule of thumb, an A indicates "impressive", a B is just "adequate", and a C indicates "many problems".

Submission

Submit a single document. Start each member's review on a new page. In each review, clearly identify both your name and the name of the person you are reviewing. (The form provided above already does both these things.) Submit to the Project Peer Review assignment on CMS.

Evaluation

You will be evaluated on whether you followed the above instructions. Especially, you must submit a full review for every member of your team, including yourself, using the supplied form.

Hints

Here is some sarcastic (and hopefully humorous) advice on how to fail your project, based on mistakes I have commonly seen year after year.