I develop machine learning models and algorithms that enable systems to interact with people and their environments in the unstructured real world. It usually means combining computer vision, robotics and natural language processing techniques with a good dose of machine learning. I enjoy building multi-modal models that combine the power of deep learning with principled known-to-work methods in robotics and vision.
Some buzzwords involved in my research: Reinforcement Learning, Imitation Learning, Semi-supervised learning, Language Grounding, 3D Vision, Meaning Representations, Transfer Learning, Sim2Real, Mapping, Planning and I guess Deep Learning.
Following High-Level Navigation Instructions on a Simulated Quadcopter with Imitation Learning (RSS 2018) Valts Blukis, Nataly Brukhim, Andrew Bennett, Ross A. Knepper and Yoav Artzi
TLDR; We build a modular, interpretable neural network architecture that explicitly addresses language understanding and grounding, mapping, planning and control, allowing a quadcopter to execute navigation commands by mapping first-person camera images to velocity commands. The model uses an in-network differentiable projective transformation to align image features in the first-person camera view with the global world-frame map representation, which is accumulated over time.
Socially Competent Navigation Planning by Deep Learning of Multi-Agent Path Topologies (IROS 2017) Christoforos I. Mavrogiannis, Valts Blukis and Ross A. Knepper
TLDR; If we can anticipate how pedestrians around us will act, we can efficiently navigate around them. We behave in a legible way by minimzing the uncertainty (entropy) of the distribution over future pedestrian trajectory topologies and thus avoiding confusing behavior. Topologies are defined using braid theory and predicted with a seq2seq model. [PDF]
Waiterbot (1st Runner-up, Tech Factor Challenge 2014-2015)
Waiterbot is a hand-built prototype for a fully featured restaurant robot. We built it in a team of 4 enthusiasts (Azfer Karim, Jagapriyan, Mohamed Musadhiq and me) for the 2014-2015 Tech Factor Challenge competition in Singapore. It can navigate from table to table, pick up and deliver orders and even serve water. It uses a custom-built gripper to pick up cups, bowls and plates (although it struggles with crockery).
Battlefield Extraction Robot (Grand Prize, Tech Factor Challenge 2013-2014)
ABER (Autonomous Battlefield Extraction Robot) is a rescue robot that can automatically find and pick up a casualty in a war or disaster scenario, where limited remote operation is possible. It's built on ROS and uses a unique conveyor-belt pickup system that inflicts minimum damage in case of bone fractures. This project earned me the DSTA Gold Medal for the best final-year project in NTU, School of Electrical and Electronic Engineering.