Project Suggestions

• Deep learning project. Be creative in finding a new learning challenge. Some useful pointers: interesting start here with a related post here, and for quite a bit more detail see here.

• Deep learning for the game of Hex (here and here, with code here (open source)). Hex is a rather interesting board game with some of the flavor of Go but easier to study and model. Also, for smaller size boards there are known optimal moves.

  Can a deep learning approach get close to optimal? What about game tree Monte Carlo search (UCT)? Insights into these questions should provide insights into AlphaGo.

• Natural language models and commonsense. Consider GPT3 or the previous version GPT2. How good is the model? Does it display some basic level of commonsense? Does it capture some basic level of rationality? Come up with a creative way of studying these questions.

• Ultimate Tic-Tac-Toe. See here . Can it be learend using deep RL? Evaluate multiple forms of play. Deep-learned features?

• Deep RL on games usually implicitly hardcodes the rules of the game (i.e., what are valid moves). Can the rules be learned too? This question is more subtle than it may appear. For example, even for tic-tac-toe it may not be easy to learn what is a valid move from just observing games being played. Investigate!

  At a higher level, this question comes back to the discussion we had around Judea Pearl's position that causal knowledge may need to be explicitly encoded and can't just be learned (efficiently) from seeing many examples. The most basic form of causal knowledge is: Given a current board position, what were the legal moves that could have led ("caused) to this position? For all interesting games, those rules are easily written down (1 page or so). However, the question is: How many example games do we need to see to learn these rules from just watching the game being played?

• A great resources for studying deep RL and games is OpenSpiel .

• Implementing Virtual Agents in a Virtual Environment

• Apply a genetic algorithm to the problem of automatic generation of computer programs.

• Apply a genetic algorithm to the problem of learning natural language grammars.

• A theorem-proving system for some (small) subset of mathematics.

  I have some pointers on how to start on this. Email me if you're interested. The key question is what parts of mathematics can be automatically derived and can deep learning push us further?

• If you're a student from a non-CS discipline, consider an problem in your domain and how AI might help. I'm happy to discuss ideas.