Ajay U. Mandlekar – Research

Research Demos and Highlights

Human-in-the-Loop Task and Motion Planning

Human-in-the-Loop Task and Motion Planning (HITL-TAMP) is a novel system that selectively gives and takes control to and from a human teleoperator. HITL-TAMP combines the benefits of imitation learning and TAMP to solve contact-rich and long-horizon manipulation.

MimicGen

MimicGen is a system for automatically synthesizing large-scale datasets from a small number of human demonstrations by adapting them to new scene configurations, object instances, and robot arms. This enables scalable imitation learning with minimal human effort.

RoboTurk

The development of the RoboTurk platform was a significant part of my PhD. The video below showcases some highlights of what we've been able to accomplish using the platform.

robomimic

I led development on this framework for robot learning from demonstrations. It offers a broad set of demonstration datasets collected on robot manipulation domains, and learning algorithms to learn from these datasets, in the hopes of facilitating progress in this important field. See the project website for more information.

Visuomotor Policy Learning on Real Robots

This is a closed-loop visuomotor policy operating at 20 hz using an action space consisting of an operational space controller. It was trained using a handful of human demonstrations collected with RoboTurk. See our work on Generalization Through Imitation for more information.

robosuite

Simulation framework for robotic manipulation. Features include several robot models, controllers, and benchmark tasks, procedural generation of tasks, multi-modal sensors, and support for human demonstrations. See the website for more information or try the code out yourself.

Selected Talks

Large-Scale Human Supervision for Learning Robot Manipulation

  • [April 2021] Talk on RoboTurk presented at NVIDIA GTC 2021.

Algorithms for Learning Robot Manipulation through Human Imitation

  • [January 2021] Talk that covers recent imitation learning algorithms for learning from robot manipulation datasets collected by humans.

Patents

Methods and systems to remotely operate robotic devices (filed)

Inventors: Ajay U. Mandlekar, Yuke Zhu, Animesh Garg, Silvio Savarese, Fei-Fei Li

Media Coverage

Thumbnail 

RoboTurk: A Crowdsourcing Platform for Imitation Learning in Robotics

Tech Xplore, November 21, 2018

“In the future, RoboTurk could become a key resource in the field of robotics, aiding the development of more advanced and better performing robots.”

Thumbnail 

Robots Learn Tasks from People with Framework Developed by Stanford Researchers

Stanford News, October 26, 2018

“With a smartphone and a browser, people worldwide will be able to interact with a robot to speed the process of teaching robots how to do basic tasks.”

Community Service

Workshop Organizer

Peer Review

  • Reviewer for NeurIPS, ICML, RSS, CoRL, IROS, CVPR, IEEE T-RO.