CS 514 Spring 2007: Assignment 2

Due: March 15, 2007.

Objective

When you download music and videos to your computer, you are typically running a web-browser or a web-services based client application that connects to a data center operated by a company like Apple, that stores the music. To gain experience with the web services technology, our goal in this assignment is to implement a client system with a sensible user interface that communicates with a web services application that serves up the music and also tracks the status of your account.

You may be familiar with some of the very fancy technologies for high-speed download that have become popular in recent years, such as BitTorrent (a system so new that the textbook doesn’t even mention it, yet growing explosively in popularity). For this assignment, we aren’t looking for anything so fancy. The download should just be a file transfer from the web service system.

However, our assignment does have an optional extra-credit component. To improve performance by supporting load-balanced operation, and to gain fault-tolerance, you can implement a replication scheme that clones the server systems. One way to approach this problem is to implement a solution of your own from the ground up. A second option is to use a technology such as a 3-tier database architecture, or the Tempest replication tool created by Tudor Marian right here at Cornell.

To obtain minimum credit for this assignment, implement a single-server solution. However, if you also implement the optional part, we’ll partially erase a weak grade on some other assignment. For MEng credit, you must implement the optional part.

Details

Actually, that’s it. We mentioned on day one that many assignments in CS514 are underspecified, because in the real world, you’ll often encounter scenarios in which something needs to be built, the rough functionality is clear, and the details are completely up to you. In this particular case, of course, you’ve probably used one of the download services and hence have at least some idea of how these things look. This is already a big help, compared to inventing solutions completely from scratch!

What to hand in

Similarly to assignment 1, we’re expecting to see two PDF files, one containing a writeup that briefly explains what you did and how it works, and one containing your code. The writeup can be a bit longer if you build a replication technology of your very own, but this isn’t necessarily a good idea. Given that there are pre-existing options such as the use of a load-balanced 3-tier database architecture (mySQL supports this), or the Tempest replication solution, you would need a good reason to build one of your own. Accordingly, if you go that route, please include a short justification for your decision.

As part of your writeup, explain how you tested your solution for correctness and to confirm that it has no performance or other functionality problems. If you implemented a replication solution, discuss your approach to validating the load-balancing functionality. Additionally, explain how your solution handles billing (depending on how you solve the load-balancing problem, downloads for a single user could be handled by different servers at different times, potentially leaving billing records scattered throughout the data center.

Performance is not a major goal for this assignment, but you are welcome to brag if your solution is particularly rapid!

Figures and other forms of graphics illustrating your work are especially welcome, and the adage “a picture is worth 1000 words” is probably true here. Relevant examples could be:

(i) Single-server case: request completion times when under very little load (say 1 request per 10 seconds), and the same when under heavy load (say >100 requests per sec) and concurrent requests from multiple users.

(ii) Replicated solution: “proof” of load-balancing and fault-tolerance. Examples:

a. show the load on the different replicas vs time (the closer the loads, the better)

b. show effect of replica-failure(s) on request completion times, to demonstrate that the system continues to function effectively even after a failure.

How to turn in your assignment

Upload your files, appropriately named, onto CMS. Please don’t email your submission to Professor Birman or the TA’s!

How we’ll grade the assignment.

The TA’s will be grading using roughly the same criteria as were described for assignment 1. As in that case, they will ask for demos of some assignments, picking “victims” at random and including additional requests in cases where something puzzled them when they reviewed the writeup and code.

Students who implement the additional functionality (replication and load-balancing) will be graded on a different “curve” than students who only do the basic functionality. So, all students will receive a grade in the range [0..100] but students who do the extra functionality will see their solution count for extra credit when we make up course grades, and also count towards MEng project credit (if you are pursuing this).

Hints.

Sorry, no hints on this one! Well, maybe one small suggestion. All else being equal, don’t assume that the very best solutions will all be ones developed from the ground up! In the real world, elegance, re-use of available technologies and avoiding proprietary (home-brew) solutions are all critical requirements. A company that proposes to build something from the ground up will probably end up needing to charge far more for a project that a competitor that manages to get the desired functionality by gluing together existing technologies in a novel way.

The same goes for students in CS514. There is a natural tendency to want to show off your hacking skills, and it is common to feel that if we learn about a technique (say, Byzantine replication) in class, the best way to really internalize it is to build one of your own. Perhaps this is true, if the only goal is to learn the technique inside-out. But CS514 isn’t only about learning techniques – we also want you to learn to make good architectural decisions. Thus if you decide to build something from scratch, you need to have a good justification (such as “Tempest was just too slow for me”)!