In CS5412 everyone will do a few individual software development homework assignments. The goal is to gain some hands-on experience with cloud ideas. These are during the first four weeks of the semester.
We want you to work on your project through the entire semester, but the actual implementation work starts in week 5 and runs to the end of the semester. So starting on time is important. About half of our recommended projects are based on cloud support for intelligent agriculture (smart farming) ideas. We would consider allowing a few teams to work on topics not in our list, but we expect that almost all the teams will actually do projects from the list. The other half are ideas you bring us, that we vet and perhaps negotiate, but where you own the whole story. Every project must align with CS5412 topics and material.
Projects are usually team based. Although there are always a few individuals who end up doing projects on their own, CS5412 really encourages teamwork. Most teams will include 2 or 3 students from CS5412. In recent years we have allowed a few projects to work in "precision agriculture teams." These teams will generally have two additional collaborators from the Cornell College of Agriculture and Life Sciences -- students specialized in dairy farming who can help the team do something real and valuable and "valid" in the sense of making choices that are realistic. We will introduce you to these collaborators. Because the dairy class is small, only a few dairy farming team projects will be possible in Spring 2021.
Every project needs a mix of real-time data with other kinds of data sets. Azure has many additional data sets, focused on things like climate and weather prediction, river depths and flood levels and flood predictions, air pollution, farm crop choices, and so forth, and the data will cover many years. But keep in mind that CS5412 isn't about data mining on static data -- every project is about something active, that deals with the outside world and has to cope with real-time events and real-time decision making, failures, consistency challenges, etc. You will need to automatically pull streaming data into your project, and combine it with other data sets in intelligent ways. The hackathon teaches some of this, but tends to focus on AI and data mining scenarios, often using preloaded Azure data sets. As a result, even if the hackathon teaches you exactly the right things on the AI and data mining side (and it generally does!), you will still need to write code to integrate those mechanisms with your streaming input.
Everyone should select a project during the first four weeks of the semester, and we will ask you to upload a plan at the end of that period, including (1) team members, if you work with others, (2) the specific project you will do, (3) your timetable for starting to show some hands-on experience with the key elements, (4) the split of tasks within your team.
You can come to us with a team proposal of your own, if you know other people in the class. You can use Piazza to meet team members through their matchmaking feature (a bit like a dating app). Or you can ask us to assign you to a team. Any of these options will work.
Plan to spend 4-6 hours per person, per week, on these projects. We do homework in the first weeks, then the project, and the only exams are in-class quizzes that take a few minutes per week and a final, so the workload is actually pretty much average for a Cornell class, provided that you start on time.
To be acceptable, the project must be matched to your group size, not too ambitious or too easy, and must have a plan to use continuously streaming data plus data already in Azure, combined within the application. You could certainly consider using Cornell's Cascade service as part of the solution, but this is not required.
Some CS5412 students have historically taken an additional semester of CS5999 credits with Professor Birman, towards their required MEng project credit. The CS5999 rule is that for each credit you take, you must put 2 hours per week in on the effort, and this adds to what you would have done for CS5412 in the first place, so it is a substantial extra effort and we expect to see a substantial "delta" to justify this.
Details are discussed in the recitations (and first lecture in the main class) but the idea in a nutshell is that you select a project for CS5412 that would be more ambitious than what you might normally have done. If you do a project with other students, they all would be doing CS5999 for the same number of credits as you -- every one puts in equal effort, and this means that a 2 or 3 person effort would be a pretty significant system (some people have prototyped startups this way! A typical big example can be found by hunting for the news releases for "Remember Me", an idea for a startup tied to CS5412 a few years ago).
We generally expect that CS5999 projects would be unique and original ideas, not just something we suggest, although there are sometimes hard open tasks in Ken's research project that a CS5999 can tackle. Even if we did provide an idea, the breakdown of how to tackle it is your job, and we expect it done well!
At each step of the project effort we will be checking that your vision really justifies the extra credits, and that you really are doing several hours per week more than if you weren't signed up for CS5999 too. Very often, the demos of these projects would occur after the normal demo day, to give a bit more time. But they need to complete by the day Professor Birman hands in his grades (there is a deadline for him), or you would end up with an INC, which we prefer to avoid.
Project grading is just the same as normal grading for the class and reflects the prelim grade, not just the project demo. This is because a project in CS5412 needs to be in part a proof that you mastered the ideas of cloud computing, and understand issues like cloud fault-tolerance, consistency, scalability, availability, existing frameworks, etc. We test your knowledge of that in part on the prelim.
All projects will have similar phases: you form a team, the team picks a project, the team develops a plan, and then the team does some preliminary work to get hands-on familiarity with the technology. Then you report on progress and on the rest of the plan, mid-semester. If working with a CALS team on a dairy topic, the idea will be reviewed by dairy professors, too, not just CS people. Then you do the harder part of the project (the full version), and then you demo what you did.
Every project ends with a presentation from a short slide set or a poster-formatted single slide, and with a demo. You will sign up for a Zoom slot, your whole team will be there, and we will see the work and ask questions and watch you run a live demo.
Cornell is organizing a spring hackathon with prizes. This year's topic will be digital farming, and the emphasis is on data mining. Now, on the one hand, this isn't a direct fit with CS5412, where we focus on technology. Still, learning to use these tools is really an amazing opportunity. This is why we award extra credit for the hackathon.
BOOM
This is a project fair that runs at the very end of the semester. You compete to show your CS5412 project there, and if selected, you get extra credit (provided that you show up and participate of course).