CS5412 Spring 2016 Project

Part I: Overview, Project concept, topic selection.

Every student in CS6412 is required to do a large hands-on project that will involve many hours coding and debugging.  This is not optional and you should not take the course if you won't have time to do the work.

Most projects will be done in teams of 4.  Due to course drops, some teams might drop to 3 later and we even allow teams to merge, so two teams that are down to 2 can merge later. 

We expect people to form their own teams, using friends or searching on Piazza.  Teams will often have people with different talents.  Part of the idea is to learn to work with people who might not all be at the same level in terms of coding style and skill, interests, etc.  The best work might involve a cloud computing core "wizard" but also a person with a love for human user interface design and two other people who are good coders but not particularly specialized in cloud stuff.  So when forming teams, don't worry about whether everyone is a top coder.  Look for people you can work with for a few weeks, who might be fun to spend time with, and don't think in terms of grading each other and picking the straight A candidates.  Instead, aim for a balance of skills, styles, and perspectives.  In industry, after you graduate, this is how people work professionally.  Learn to be a professional starting this semester!

Each project will have a fairly rigidly defined cloud computing data repository that supports reads and updates and can scale well.  You'll be using a tool for this that we call the Vsync software library.  It offers fault-tolerance, consistency and other guarantees.  One of the project members will certainly need to work with it, and it helps if this person is familiar with C#, although C++/CLI, Python, Ruby and F# are also options.

The projects will also have a "front end" that deals with the person using the system.  For this, web "app" building skills pay off.   Most projects will be portable but will have a primary target platform: perhaps Android, perhaps the iPad, perhaps a browser.  So every team will need a project member who can have primary ownership of this side of things.  Again, we won't be teaching you how to build this part, so you'll need to be kind of self-motivated.  Prior GUI experience really pays off here: the app-building platforms are very much like GUI-building platforms.

Last, there will be a middle part of your system that implements the main application logic.  The remaining two project team members will work mostly at this level, and on performance testing.

The functionality of the project needs to be something cool, that can have a nice front end, and that works with data captured from the user or from the device, and where sharing data via the cloud is beneficial.  Some ideas for possible projects (but feel free to come up with your own):

No matter what, think in terms of "How will my system look to its users?  What API will they see?  How will it benefit them?".  You may want to think about privacy or other considerations. 

Then ask "What data should my application capture, and how will I get that data?".  This becomes the data you will be writing to the cloud storage system.

Next ask "What should the cloud platform compute on behalf of the collection of users?"  With large numbers of users, maybe millions, the cloud might be able to do many things the individual systems can't do on their own. 

Finally ask:  "How will the user platform read the data from the cloud, and what will it do with it?" 

Beyond this, we expect you to also think about performance, scalability, security and even fault-tolerance and real-time responsiveness requirements.  We'll grade you in part by the quality of your thinking about these issues and the degree to which you show us that you really applied what we learn in the course to the design of your project!

BOOM CS projects fair.

We strongly encourage you to showcase your work at BOOM even if the project is not 100% finished by then.  Think of BOOM as an early deadline for having something amazing working.  Then in the final few weeks of the semester your already working project can become a totally awesome one.  But by finishing something for BOOM, the pressure will be way lower for the rest of the semester!  Any project shown at BOOM automatically gets extra credit of 10 points (our of 100).  We know that it is hard work to reach that point and we want to reward you for doing so!

Single-person projects

 Every semester, one or two people talk us into letting them do a less ambitious project on their own.  But after Cornell, working in teams is going to be a part of your life.  We really strongly encourage you to use this as a chance to find ways to work with other people and to contribute to something that is bigger than your own part.  This isn't the same as socializing or making close friends: people can work together professionally without ever becoming pals.  So there is a skill to be gained here, and your CS5412 project is a chance to develop that skill.  Use Piazza to find other team members if you don't know people in the class.

Projects that you wish to also count for MEng Project Credit.

As noted earlier, you can use your project for MEng credit.  If you do this, we expect the entire project to be a reasonably ambitious one, and we want every team member to be using the project for CS5999 credit.  In effect, MEng projects are just like other projects, but you will work more hours on the project (all of the team members will do so), so can do something a bit fancier.  Your project grade becomes your CS5999 grade.

If you plan to sign up with Professor Birman for CS5999 credits, please arrange to meet with him for approval first.  He'll want to make sure you have the prerequisites for these projects.  For example, you do need experience using the kind of software platform the project requires.  You'll work pretty hard for your 3 credits, so he'll also want to be sure you actually have time for what you are proposing to do.

Notice that we are not suggesting other kinds of projects, outside the framework outlined above.  MEng project or not, every team will have a similar set of goals customized mostly in terms of the front-end behavior.

Part II: Details of Project Organization

How CS5412 Projects Work:

As the overview will have made clear, our main goal is that by the end of CS5412 project you should be capable of designing a scalable, fault-tolerant web application. Do not worry if you do not know what this means on day one: the whole point is that we plan to teach you the needed technologies so you'll learn what you need to do after the class begins!!! We also want every group to work on some application that feels useful and innovative.

The work occurs at a steady pace over the semester.  To this end, we break the project into five Phases.  We want to help if you get into trouble, so progress is going to be checked in each one of them. Any feedback that you get for an intermediate Phase (not the last one) should be constructively used in the next one. You can see more details about the different phases below.  Grading is based on the outcome from the whole semester, not the mid-point checking.  So there is nothing embarassing or harmful about needing help from us at one point or another.  We've designed this structure to optimize your learning experience and to be able to provide you with guidance when you need it, with no grade penalty of any kind at all for needing it.

Phase 1: Team Formation and Project Idea (11.59pm, Sunday 31 January 2016)

In this phase, you should form teams of 3-4 students. You should really aim for 4 unless you cannot find one more student. If we find that more than 20% of the teams are composed from 3 people (or less) we will be forced to change some or even all of them ourselves (Hint: Use Piazza to form your team if you cannot find collaborators). The team should submit a report (~1-2 paragraphs) which should contain the following:

  1. Project title. [Title]
  2. List with the students (name, netid) participating in the team. [Authors]
  3. Brief description of the project you intend to do. [Abstract]

A few days after submission, you should get approval to continue. Otherwise, you should agree with the TAs on a different project idea.

Phase 2: Project Proposal (11.59pm, Thursday 11 February 2016)

In this phase, you should explain in more detail what you are planning to do. Furthermore, you should try to link your idea with Cloud Computing. Finally, you should make a plan for the next phase. The team should submit a report (~1-2 pages) which should contain the following:

  1. All the previous content with possible improvements.
  2. Motivation behind your project. Why is the application useful? What are the challenges from the Cloud Computing perspective? [Introduction]
  3. Implementation design (for the following phase). How you are going to curry out the completion of the next phase? What interface are you going to provide? What is the general architecture for your proposed solution (figure is recommended here)? [Implementation]
  4. Experimental plan (for the following phase). How are you going to show that you have achieved your goals? What experiments you will curry out in order to evaluate your implementation? [Evaluation]

A few days after submission you should get feedback about this phase. In case we propose some improvements, you should make sure to consider them and include them in the next report/phase. In case we find the report problematic, you will be called to discuss about it.

Phase 3: Intermediate Report I (11.59pm, Thursday 10 March 2016)

In this phase, you should implement the interface for your application (client-side), a background server that executes the commands of the users and implements the interface of your application correctly (back-end server-side), and a relay server that is going to accept web requests and forward them to the background server (front-end server-side). You should make sure to evaluate your implementation by performing experiments that show you have achieved your application's goals. The team should submit a report (~3 pages) which should contain the following:

  1. All the previous content with possible improvements (leave out the previous plans).
  2. Implementation design (for the following phase). How you are going to curry out the completion of the next phase? What is the architecture for your proposed solution (figure is recommended here)? [Implementation]
  3. Evaluation so far. Show all the experiments that you have done with the current implementation. [Evaluation]
  4. Experimental plan (for the following phase). How are you going to show that you have achieved your goals? What experiments you will curry out in order to evaluate your implementation? [Evaluation]

A few days after submission you should get feedback about this phase. In case we propose some improvements, you should make sure to consider them and include them in the next report/phase. In case we find the report problematic, you will be called to discuss about it.

Phase 4: Intermediate Report II (Scalability) (11.59pm, Thursday 14 April 2016)

In this phase, you should make your system scalable. This means that instead of one back-end server, you should use n (optionally you can do the same with the front-end server). As n gets larger the performance of the system should become better (served requests per second, average latency, etc.). The team should submit a report (~4-5 pages) which should contain the following:

  1. All the previous content with possible improvements (leave out the previous plans).
  2. Implementation. You should make arguments that show that your application is scalable for large n (based on your current architecture and how you divide the load between servers). [Implementation]
  3. Implementation design (for the following phase). How you are going to curry out the completion of the next phase? What is the architecture for your proposed solution (figure is recommended here)? [Implementation]
  4. Evaluation so far. Show all the experiments that you have done with the current implementation. You must have an experiment that shows that your application is scalable (you should at least compare 1 vs 2 servers). [Evaluation]
  5. Experimental plan (for the following phase). How are you going to show that you have achieved your goals? What experiments you will curry out in order to evaluate your implementation? [Evaluation]

A few days after submission you should get feedback about this phase. In case we propose some improvements, you should make sure to consider them and include them in the next report/phase. In case we find the report problematic, you will be called to discuss about it.

Phase 5: Final Report (Fault-Tolerance) (11.59pm, Tuesday 10 May 2016)

In this phase, you should make your system fault-tolerant. So far, we were not worried about failures in computer instances that frequently happen and can ruin the functionality of the system. Assuming fail-stop failures (if you do not know what this means, be patient and you will learn it in this class), you should make your system tolerate t of them (optionally you can tolerate less benign failures). This means that your system should continue working normally as before even when t instances stop working. Fault-tolerance might add an overhead to your application but you should not have a large decrease in performance compared to the previous phase. The team should submit a report (~6 pages) which should contain the following:

  1. All the previous content with possible improvements (leave out the previous plans).
  2. Implementation. You should add arguments that show that your application is fault-tolerant for t (based on your current architecture and how you handle failures). [Implementation]
  3. Evaluation. Show all the experiments that you have done with the current implementation. You must have an experiment that shows that your application is scalable (scalability has not been compromised by adding fault-tolerance) and a corresponding one for fault-tolerance.[Evaluation]
  4. Summary. Summarize the inovations and accomplishments of your team.[Conclusions]

A few days after submission you should get your grade.

Project Demo: (11-12 May 2016)

You will have two days to book a slot for a 10-minute demo with the TAs. In these 10 minutes you should be able to exhibit the application's interface and show that it works correctly even when 1 failure occurs (fault-tolerance). You should also show that your system can work efficiently with at least 2 back-end servers (scalability, load-balancing). All the team members should be present.

We may call you for a demo within the semester. At least one member should show up and show the functionality that the system should have according to the latest team's report. You will have two days to schedule an appointment after we call you. We will have multiple slots available in these two days so it should be easy to find a good time. If you cannot make it once, your grade will suffer mildly. If you cannot make it more than once, there is going to be a significant penalty to your grade.

Format for Reports:

All the report should have the USENIX format. In each phase, all the deliverables have a section marked with brackets. This section should contain the corresponding material.

CMS:

All the documents excluding the poster need to be commited to CMS.

MEng Project Credit:

At the end of Part I, we explained that if you wish to obtain MEng project credit, you can do so by signing up for CS5999 in addition to signing up for CS5412.  You can't do just CS5999 by itself.  But if you do this, the only real impact is that we expect to see that your project is reasonably ambitious and that you are doing it in a serious, engaged way.  After all, you are getting extra credit this way!  But you get the same grade as you normally would have gotten.  For CS5999 your grade would be the project part of your CS5412 grade (e.g. exams do count towards CS5412 but not towards CS5999). 

Due Date: 

CS5412 projects are due on the last days of the course, which are set aside as project demo days.  On request, short extensions of at most 10 days may be granted, but you must request the extension, explain precisely why you need extra time, and get actual permission from Professor Birman or a TA, in writing.  Otherwise, late projects will be reviewed during the same 10 day period but if you didn't get permission to finish late, a penalty to your grade may apply (e.g. A+ work might get an A grade if you finished a week late and didn't have permission to work a week longer).

Grading: 

Your MEng project will be graded by doing a demo that shows what you did to the grading team composed of Professor Birman or the TAs.   We grade in the range B to A+ for most projects.   Sometimes a very weak effort may receive a B- or lower.  Our aim is to have the median grade be on the B+/A- border: half above and half below.  CS5412 grading will also reflect grades on the prelim and final. 

To get an A+ in CS5412 you must get great exam grades and also have one of the very best projects that the TAs saw.  We award very few A+ grades.   Sometimes we don't award any; more often, four or five students in the entire class might receive an A+. 

Extra credit: 

As noted in Part I, any MEng project shown at the BOOM projects fair will recieve extra credit (e.g. B-level project work might receive a B+ project grade).  However, extra credit will not boost your project grade from A to A+.   For BOOM you need to prepare a poster and be able to do a demo.