Dipendra Misra

Doctoral Student,
Department of Computer Science,
Cornell University
Advisor: Yoav Artzi

Addr: Cornell Tech,
2 W Loop Rd,
New York, NY 10044

After a great summer at Microsoft Research Redmond, I am excited to be at the new Cornell Tech Campus.

Update: Tutorial on Markov Decision Process Theory and Reinforcement Learning [Part 1] [Part 2] [Blog]

Research and Publication

Research Interest: Situated Natural Language Understanding, Semantics, Semantic Parsing, Dialogue Modelling, Reinforcement learning.

I develop machine learning models for mapping natural language to meaning representations (e.g., Misra et al. 2016). I am particularly interested in natural language grounding problems e.g., robot natural language understanding (Misra et al. 2017, 2015a, 2015b, 2014), conversation modeling etc.

Conference and Workshops

Reinforcement Learning for Mapping Instructions to Actions with Reward Learning (NCHRC, AAAI Fall Symposium 2017)
Dipendra K. Misra and Yoav Artzi
Punchline: Learn a reward function directly on images for mapping natural language instructions to actions.
[Paper] [Code]

Mapping Instructions and Visual Observations to Actions with Reinforcement Learning (EMNLP 2017)
Dipendra K. Misra, John Langford and Yoav Artzi
Punchline: We present an approach for mapping natural language instructions and visual observations to agent actions using reinforcement learning.
[Paper] [Code] [Arxiv Preprint]

Neural Shift-Reduce CCG Semantic Parsing (EMNLP 2016)
Dipendra K. Misra and Yoav Artzi
Punchline: We present the first published shift-reduce parser for CCG semantic parsing. Our novel learning algorithm considers learning without annotated parse trees, with type constraints and complex categories.
[Paper] [Supplementary] [Code]

Environment-driven lexicon induction for high-level instructions (ACL 2015)
Dipendra K. Misra, Kejia Tao, Percy Liang, Ashutosh Saxena
Punchline: Learning new concepts and meaning of new verbs at test time using environment as an implicit signal.
[Paper] [Supplementary] [Code] [Data] [Simulator] [Bibtex]

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions (RSS 2014)
Dipendra K. Misra, Jaeyong Sung, Kevin K. Lee, Ashutosh Saxena
Punchline: Grounding language to actions with application on PR2 robot. Learn from users playing with simulations.
[Paper] [Website] [Simulator] [Bibtex]


Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions (IJRR 2015)
Dipendra K. Misra, Jaeyong Sung, Kevin K. Lee, Ashutosh Saxena,
[Paper] [Website] [Bibtex]

Other reports, articles etc.

Robo Brain: Large-Scale Knowledge Engine for Robots (ISRR 15)
Ashutosh Saxena, Ashesh Jain, Ozan Sener, Aditya Jami, Dipendra K. Misra, Hema S Koppula,
Punchline: We introduce a knowledge engine, which learns and shares knowledge representations, for robots to carry out a variety of tasks.
[Paper] [Website] [Bibtex]

Bio and Employment


  • Amazon AWS Research Grant (2016)
  • Cornell University Fellowship (2013)
  • 2nd rank in the 2013 graduating B-Tech batch of IIT Kanpur
  • OPJEMS Merit scholarship (2011-12 and 2012-13)
  • Invited to present a lecture in the 2012 Annual Meet of Calcutta Logic Circle celebrating the Alan Turing year
  • Invited to speak at the 10th Indo-German Winter Academy on High Performance Computing
  • Academic Excellence Award (2009-10 and 2010-11)
  • Was ranked 156th in first attempt out of 3,50,000+ students who appeared in JEE 2009
  • Certificate of merit in National Standard Examination in Physics 2007(State Level)


I like spending some time as an amateur pianist (some of my Chopin recordings- Ballade 1 Op 23 [Part 1/2], Valse Op 64 2, Raindrop Prelude, Valse Op 34 2, Posthumous Valse A minor, Nocturne 20 C# Minor), reading, writing and composing my thoughts. I like reading about history and politics.