CS 4701: Practicum in Artificial Intelligence

Enrollment Questions

Most questions about enrollment are not specific to this course and you'll find the answers at:

• Enrollment information for CS classes
• Waitlist FAQs.
• Information specifically for 4000- and 5000-level courses.

If, after reading the above resources, you still have a question about getting into the course, please file a help ticket at https://tdx.cornell.edu/TDClient/193/Portal/Home/. Thank you!

Course Staff:

• Instructor:
  • Prof. Kevin Ellis, Gates 442B
• Course Administrator:
  • Lacy Jordaens, lsl92@cornell.edu
• TAs:
  • Albert Tseng
  • Snehal Bhagat
  • Diyang Li
• Office Hours can be found at https://calendar.google.com/calendar/u/0?cid=NjdmMmVjMzViNTNhOTc0Zjg2YzI4YzA3NzJmZTYyZTkxN2M4YWI3YTkzMmRkYTY2NjE3MmVkNTRkZTc2ZjhhYkBncm91cC5jYWxlbmRhci5nb29nbGUuY29t.

Course email

• Please always put "CS4701" in the Subject field of any course email.
• All course-specific administrative matters: lsl92@cornell.edu (Lacy Jordaens) - such as, for example, special accommodations letters, including those arising from COVID quarantine events.
• All course content matters: If you have any course content matters please contact the instructors via Ed Discussion.

Orientation lecture

CS 4701 online

• Canvas: The Canvas website for the course will be used as the entry point to all online material for the course, and will include links to the other online resources for the course.
• Ed Discussion: Online course discussions will take place on Ed Discussion, and is accessible via the course's Canvas page. You will be automatically enrolled in it. You are required to read it - it will be the primary place where we send information to students.

Prerequisites:

Officially this course is listed as a co-requisite of CS 4700. This unfortunately is misleading. It is not taught in sync with 4700 but rather is a stand alone course where you have the opportunity to do a semester-long project in teams of 3 on a project in AI of your own choosing. It is strongly recommended that you only take 4701 if you have already completed CS 4700 or any other AI course or otherwise have had a substantive introduction to AI. Without that students can't "hit the ground running" at the start of 4701 and simply don't do well in the class. To be super clear: You do not have to have taken 4700 to take 4701. But you do need to have some body of AI knowledge that you've acquired before taking 4701.

Grading:

• Nearly the entirety of your grade is based on your project. Your overall grade is determined as follows:
  • 10%: Completing all milestones as requested.
  • 10%: Short presentation at the end of the semester.
  • 80%: Your overall project, computed as follows:
    • 50%: Effort
    • 20%: Clarity of writing
    • 30%: Evaluation
  These are discussed further below.
• Late submissions will be penalized 1% for every hour they are late. Thus, for example, after 50 hours your grade for that component would be decreased by 50%. If you have extenuating circumstances, please send email to the course administrator, Lacy Jordaens, lsl92@cornell.edu. We will provide extensions when applicable for injuries, family emergencies, COVID quarantine, etc., on a case-by-case basis. Getting confused about the time an assignment is due is not an extenuating circumstance.
• Students should work in teams of 3. Any variance from this number requires prior permission and is rarely granted.
• There is usually a start-of-semester partner finding event which can help you find partners for your group project. If this is happening as well this year we will announce it ASAP on EdDiscussions.

Milestones:

• Milestone 1: Teammate Selection
  Due: Thursday, February 8, 11:59pm
  
  
  ONE MEMBER of your team should go to Canvas and upload a file containing your signed Team Agreement (see below) under the Teammate Selection assignment.
  
  TEAM AGREEMENT Working together as a team with an entire course grade depending on it is a serious commitment to your teammates. To reflect this commitment, we are asking that you take a moment to reflect on it by submitting a "Team Agreement". It may be copied verbatim from what follows, or even better it can be adapted to reflect your further thoughts on working together as a team. You should reproduce the agreement together with your names and each team member should sign it (electronic signature is fine), indicating acceptance of these expectations and your intention to fulfill them.
  
  
  • We all show equal commitment to our objective.
  • We will all take part in deciding how work should be allocated.
  • We are committed to helping each other learn.
  • We will acknowledge good contributions from team members.
  • We will handle disagreements and conflicts constructively within the team.
  • We will give constructive feedback to one another, and will receive feedback in that light.
  • We will attend all planned team meetings and stay for their full duration.
  • We will keep each other fully informed about what is going on.
  • When one of us is under pressure, we will offer to help that teammate.
  • We will support each other both inside and outside the group.
  
  If you do choose to modify this agreement, make sure the expectations that you list are thoughtful and realistic. For example, "We will get a perfect score on the project" would not be meaningful as a principle to follow for working together. If you'd like further thoughts on writing your team agreement, please consult the CS3110 Fall 2019 Teams materials prepared by Prof. Michael Clarkson.
  
  
• Milestone 2: Project Proposal
  Due: Friday, February 16, 11:59pm
  
  
  ONE MEMBER of your team should upload a copy of the proposal to Canvas.
  
  Your proposal should have the following information, in the order specified:
  • Title of your project
  • Team member names with Cornell NetIDs
  • AI Keywords: What area of AI does your project involve? For example, reinforcement learning, natural language processing, game playing, etc.
  • Application setting: To the extent relevant you should specify the "application" task you are attempting. For example, for a game playing program this would be the name of the game, for a chatbot it would be the setting it would be in, etc.
  • A clear and concise description of what you plan to do:
    • Section 1: (1-2 paragraphs) What you want to do. Give us the big picture.
    • Section 2: (2-3 paragraphs) What specific aspect of AI will be at the core of your project? Show us that you've thought more than superficially about how you would go about doing what you just described in Section 1.
    • Section 3: (2-3 paragraphs) How will you evaluate your system? Make sure you have thought about this carefully. For example:
      • How will you know if you were successful in what you attempted? For example, if you're going to build a game-playing system, how will you know how good it is? Is there an existing program you will go head-to-head against? (And if so, think about this carefully - it is easier to go head-to-head with an open-source game playing program that you can download and run locally than to do so with a game that you can only play via, say, an iPhone.) Or will you have it play against humans? (And if so, think carefully about this - how will you know how good the human players are, to calibrate your results?)
      • If you plan to explore different solution approaches, how will you determine their relative merits? Or if you had different design options for some facets of your system and you explored the various options, how did you settle on the ones you ultimately used?
      One special word of warning: If your evaluation involves people interacting with your system you should recruit people other than yourselves, such as other 4701 students. Plan for evaluation, and include it in the next item, the project timeline.
    • Section 4: A timeline for your project, spanning implementation, evaluation, and writing. Take this seriously. Show us that you thought carefully about what work you'll be doing over the semester. It should not simply be three items, "implement", "test", "write up". What milestones are involved in the implementation? What can be doing in parallel versus must occur in sequence? Did you allow time to update your system in light of some initial testing you perform? You will be doing what at each step along the way? This is not a contract, and may change as the semester progresses. Rather, it is to show us - and yourselves - that you have a project of semester-long timescale and that you've thought about what responsibilities you'll each have along the way.
    • Section 5: Any existing resources (software, data, etc.) you are thinking of relying on. You do not need to reinvent the wheel. If there are software or data resources that you want to use in your project that's ok. It is the work you do beyond these that is graded. Thus, for example, if machine learning will be a component of your system, make sure the data you need is available and actually responds to your needs. You should not say that you will be looking for data, you should be discussing the data that you've already found. (It could very well be that your plan is to generate your own data resource, which is also fine - what you do will depend on the nature of your project. Please include separate subsections, as relevant, for software versus data resources.
    • Section 6 (If relevant:) Is your project related to anything else members of your team are doing? How? Your project can be connected to something else you are currently doing. The only requirements are that (1) the work for the project must be disjoint from what you are doing elsewhere, (2) you disclose the connection to this external activity, and (3) all parties (both this project's teammates as well as those involved in whatever else it is connecting to) are aware of its use in this fashion. It is extremely important that you are clear what is distinct in this project compared to the work you would be doing otherwise. Make sure to be specific.
    • Section 7 (If relevant): List any references (technical papers, websites, etc.) you used in formulating your project.
  
  
• Milestone 3: Status Report
  Due: Wednesday, March 27, 11:59pm
  
  
  ONE MEMBER of your team should upload a copy of your status report.
  
  

  It is natural for your project to morph over time. The goal of the status report is to (1) make sure you have been working on your project, (2) give details about project elements that weren't apparent at the time of the project proposal, and (3) give you a chance to explain how what you're hoping to do has changed. It also gives us a chance to provide feedback to help you along.

  Your status report should include verbatim the project proposal that you originally submitted and directly add to it. Please make it easy to see what was in the original proposal versus what is new text. One good way to show what is new versus what was already there is to use a different colored font for your new text to distinguish it from the earlier proposal text.

  The best-case scenario is that things are going exactly as you planned, which would make your status report trivial to prepare. But things never go exactly as planned. It is totally OK if things have changed or there's more that you didn't think of at the time of the proposal. This update will most typically include things like the following:

  • We didn't realize that doing part X was so hard, so it's taking us longer to do it and we had to scale back some other part Y.
  • We said we would do X but it doesn't seem to be working so we decided to instead do some modified version of X.
  • We discovered this great software/data resource which simplifies X, which means we're now going to try to do something bigger, X+Y.
  • We learned that the great software/data resource X that we expected to use wasn't available, so we're instead using Y.
  • We thought more about how to evaluate the project, and instead of doing X we now realize that it will be better to do Y.
  
  

  Please also provide an updated timeline. It should say which items in the original timeline have been done, and what (if anything) you are changing in the timeline given what you've learned about the project thus far. Similarly, if the planned division of labor has changed, say so. Please also add any additional references to technical papers or websites if relevant.

  We expect to have seen substantive work by this point in the semester, and it is a component of the effort portion of your grade.

• Milestone 4: Project Presentation
  Due: May 18, 4:30pm
  
  
  You will give a 15 minute recorded oral presentation that gives an overview of your project and what you accomplished. The goal here is practical: It's not always easy to understand projects solely from their reports. Your goal for tpresentation is to help us understand what your project wound up being before we then review your project report. Don't try to present all your work. Do try to present the big picture of what you were trying to do, what you accomplished, whether you succeeded or not, and how you assessed that. This should not necessarily be simply reciting the sections of your report. Although your presentation's structure might mirror the structure of your report, that isn't always the best way to convey an overview of what you did in an oral presentation format.
  
  
• Milestone 5: Final Submissions
  Due: May 18, 4:30pm
  
  

  You will need to submit three things:

  1. A final report
  2. Your code
  3. An individual report
  
  MILESTONE 5a: Project Report (Group): This is the main report for your project. ONE MEMBER of your team should upload a copy of the proposal. Your project group will automatically transfer from the Status Report assignment.
  
  

  Your final submission should be a roughly 10-page paper. As with the proposal, this is a guideline that you might very well vary from. For example, your report might be longer if you require many figures. To help you calibrate this guideline, a 5 page paper would likely be too short whereas a 15 page paper would likely be too long.

  Your report should include the following, in the specified order:

  • Title page:
    • Title of your project
    • Team member names with Cornell NetIDs
    • AI keywords
    • Application setting
    • List the names and NetIDs of any other students who played a role in your submitted project
    • For example, did others proofread your paper or serve as user study participants for your evaluation?
  • The body of your report will have two main sections of equal importance:
    • Part 1: Project description
      • Say what you had planned to do, then explain what your project wound up being.
      • Explain what were the key aspects of AI that were the core of your project.
    • Part 2: Evaluation
      • Explain what assessment you conducted concerning your project. How well did you do? Be clear about (1) what questions you asked, (2) how you went about answering them, and (3) what specific answers you got. This will likely be quantitative. Your question(s) might be about how well your system worked (perhaps along multiple dimensions), or it might be about how different elements of the system design contributed to overall success. Your project is not complete if you have not thought about how you can evaluate your accomplishments; the nature of this evaluation is part of the design of your project. Be clear why you think you were successful, or not, or in what circumstances. Sloppy or hand-wavy discussions of performance will significantly negatively impact your grade.
  • References (as relevant):
    • Technical papers, books, websites, etc., that played a role in the formulation and execution of your project
    • Software resources
    • Data resources
  • Relationship to other work by team members (if relevant): If you project was connected to other work any members of your team were involved, please summarize how this project was distinct from it.
  
  

  Note what this DOES NOT include: code listings, software architectures, etc. If you're discussing the different modules of your system your report is at the wrong level. Your report should be about the ideas. At this point in your studies I'm presuming that you can architect and code software, and what heft there is in your hacking should be apparent through the ideas and evaluation you conducted.

  MILESTONE 5b: Code:

  ONE MEMBER of your team should submit a copy of the code to Canvas as a .zip file; this is part of the same Canvas assignment as your final project report.
  
  You will NOT be graded on the quality of your code. It doesn't even matter what programming language(s) you use. However, we do want to have your code as a record of your work. This will allow us to check, for example, that your team implemented the code and that it does what you reported it does.
  
  

  MILESTONE 5c: Project Report (Individual):
  Each team member should submit on Canvas a short (1-2 page) personal report on the project. This is an individual assignment, not a group assignment. This report should have three brief sections:

  • Section 1: Outline your individual contributions to the project.
  • Section 2: Describe what personal lessons were learned from doing the project.
  • Section 3: (If relevant) If you played a role in the project of another team, such as serving as a test user or reading their report, you should document this here.

Characteristics of a Successful Project

Warnings

1. Make sure your work is focused on AI and not on ancilary things like your user interface, database back ends, etc. You will likely want to spend time on an interface, but if at the end of the semester your report is mostly discussing the time you spent on the UI you will have missed the point of 4701. AI should be front-and-center.
2. This is an AI programming course. Many recent successes in AI are powered by machine learning, and you might very well choose a project that has some significant machine learning in it. If so, make sure this is a programming project as opposed to one where your coding was largely writing scripts that invoke machine learning packages, exploring your learning algorithm's parameter space but without doing anything particularly original in terms of exploring the design and implementation of AI algorithms.
3. CS 4701 is a highly self-directed courses. There are no regular meeting times, just regular milestones. This is fantastic for some students, and horrific for others. Think about what motivates you to do the work in a class. If the idea of immersing yourself in a project without twice-a-week class meetings taking you along on a carefully crafted shared journet with other students, this is a great choice for you. If, however, if it is the regular contact points, graded assignments, etc., that serve to motivate your classroom efforts, CS 4701 is probably not a good choice for you. These statements do not impose a value judgment on one being better than others, it's simply a fact that students learn differently. Are you a student who would likely let the major work for 4701 languish until the end of the semester approaches because of the lack of constant classroom meetings, reminders, etc.? There is no way to hide when this happens, and the resulting projects turn out bad, resulting in bad grades. For some the course provides a freedom to lose oneself working on an exciting semester-long programming journey. For others it's an unstructured nightmare. Be true to yourself and take this class only if you think you have the personality to thrive in one of the few open-ended programming project classes that CS offers.

Class Policies

• Auditing: There is no auditing option for this course. Do not sign up to audit this course.
• Academic Integrity: From Cornell's code of academic integrity:

  Absolute integrity is expected of every Cornell student in all academic undertakings. Integrity entails a firm adherence to a set of values, and the values most essential to an academic community are grounded on the concept of honesty with respect to the intellectual efforts of oneself and others. Academic integrity is expected not only in formal coursework situations, but in all University relationships and interactions connected to the educational process, including the use of University resources. ... A Cornell student's submission of work for academic credit indicates that the work is the student's own. All outside assistance should be acknowledged, and the student's academic position truthfully reported at all times. In addition, Cornell students have a right to expect academic integrity from each of their peers.

  If you use external resources, you must document it - your project is what you do on top of these resources. Failure to acknowledge resources and let them be represented as your own work - either explicitly or by omission of documentation of their origin - would be an academic integrity violation.
• Accommodations There are various non-academic reasons students can face difficulties in successfully completing a course of study. Some can be short-term, some chronic. If there are any special accommodations that we must be aware of to help you be a successful student in this course it will typically be documented in a letter from the appropriate Cornell office, most typically Student Disability Services. Please email such letters to the course administrator, Lacy Jordaens, at lsl92@cornell.edu. While it is our responsibility to be mindful of relevant Cornell policies governing such matters, it is your responsibility to notify us of the relevance of these policies to you. Please submit the necessary documentation during the first two weeks of the semester, so that we can ensure that the necessary arrangements can be made as quickly as possible.
• Respect: Everyone - the instructor, TAs, and students - must be respectful of everyone else in this class. All communication, in class and online, will be held to a high standard for thoughtfulness and inclusiveness: it may never target individuals or groups for harassment, and it may not exclude specific groups. Sometimes such acts occur without hurtful intent, but nonetheless create hurtful experiences for others. One very common example of this is humor when it targets particular individuals or groups. The label on this section is "Respect" because the surest way to be thoughtful about such matters is to always interact with others in respectful ways. Humor that targets particular individuals or groups is not respectful. If any of the communications or interactions in this class doesn't meet these standards - whether with a fellow student or with course staff - please contact the instructor; if for whatever reason you don't feel comfortable discussing something directly with the instructor you may also choose to raise it with your advising office or the department chair.

Sample Projects

• AI for playing board games, computer games, card games, party games, etc., possibly using learning. For example:
  • Atari games, Super Smash Brothers, Tetris
  • Word games like "Codenames" and "the Wikipedia game"
  • Card games like Bughouse chess, Pentago, Machi Koro, Onitama, Risk, "Ultimate Tic-Tac-Toe"
  • Social deception games: Mafia, Avalon
  • Generic card game AI, where the game rules are an input to the algorithm
  • Poker (of which there are many possible variants)
  • Pokemon Battles
• Transcribing handwritten characters, either from pixels or motor trajectories. You could also try automatically translating Manga, and you don't need to confine yourself to the same writing system as is used in English.
• Creative applications of artificial intelligence:
  • Creating aesthetically pleasing geometric designs, for example through genetic algorithms. Electric Sheep can be a source of inspiration.
  • Automatically making/composing music, from lyrics such as rap rhymes to beats such as those found in electronic music
  • Automatically creating multi-cuisine recipes
• Revisit SHRDLU, expert systems, or other older classic AI landmarks through the lens of modern AI (particularly probabilistic inference and/or machine learning)
• Design and implement a probabilistic programming language for a class of artificial intelligence problems or approaches, such as computer vision or neurosymbolic architectures. For inspiration, you can check out Picture, or Edward. Use your system to formulate and solve an artificial intelligence problem, such as sentiment analysis or few-shot learning
• Investigate multiagent reasoning, such as using modal logic (Russell and Norvig Chapter 10.4) or Rational Speech Acts.
• Build a system that "synthesizes programs": given a specification of what a computer program should do, search or plan through a space of source code to find a program that satisfies that specification. Classic examples are SmartEdit, FlashFill, and works in the genetic programming literature
• Visual Question-Answering: build a system which can answer questions about a visual scene by parsing natural language into a logical expression, parsing a visual scene into discrete objects with attributes, and then resolving the query through logical inference methods
• Robot motion planning, combining discrete search/planning methods with continuous optimization/RRT, possibly hierarchically. You may find it useful to write or use a preexisting simulator for a basic robot gripper/objects it can manipulate
• Trying to solve the Abstraction and Reasoning Corpus, making what progress you can
• Equation discovery/symbolic regression using genetic algorithms. When are the pros and cons against parametric approaches?
• AI for accessible/smart user interfaces
  • Eye-swipe user interfaces, or other uses of an eye-tracker such as for typing and mousing
  • Sign language understanding
  • Smart voice-based interfaces
  • Autocomplete-like features for graphical user interfaces, such as predicting what menu item the user will click next and highlighting it
• Advice/prediction systems based on probabilistic inference and/or machine learning
  • Advisors for taking care of houseplants or pets
  • Predicting financial movements, such as cryptocurrency prices based on twitter
  • Health applications, such as advising on skin disease or diet/exercise
  • Other tasks like advising on fantasy football or business applications like customer volume
• Combining perception with problem solving
  • For example starting with an image of a crossword puzzle, parsing it into symbolic form, and then solving it