Ajay U. Mandlekar – Research

Research Demos and Highlights

DexMimicGen

DexMimicGen is a novel data generation system for bimanual dexterous manipulation. It generates demonstrations automatically for challenging humanoid manipulation tasks.

SkillMimicGen

SkillMimicGen (SkillGen) is a novel data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation.

Human-in-the-Loop Task and Motion Planning

Human-in-the-Loop Task and Motion Planning (HITL-TAMP) is a novel system that selectively gives and takes control to and from a human teleoperator. HITL-TAMP combines the benefits of imitation learning and TAMP to solve contact-rich and long-horizon manipulation.

MimicGen

MimicGen is a system for automatically synthesizing large-scale datasets from a small number of human demonstrations by adapting them to new scene configurations, object instances, and robot arms. This enables scalable imitation learning with minimal human effort.

RoboTurk

The development of the RoboTurk platform was a significant part of my PhD. The video below showcases some highlights of what we've been able to accomplish using the platform.

robomimic

I led development on this framework for robot learning from demonstrations. It offers a broad set of demonstration datasets collected on robot manipulation domains, and learning algorithms to learn from these datasets, in the hopes of facilitating progress in this important field. See the project website for more information.

Visuomotor Policy Learning on Real Robots

This is a closed-loop visuomotor policy operating at 20 hz using an action space consisting of an operational space controller. It was trained using a handful of human demonstrations collected with RoboTurk. See our work on Generalization Through Imitation for more information.

robosuite

Simulation framework for robotic manipulation. Features include several robot models, controllers, and benchmark tasks, procedural generation of tasks, multi-modal sensors, and support for human demonstrations. See the website for more information or try the code out yourself.

Selected Talks

Large-Scale Human Supervision for Learning Robot Manipulation

• [April 2021] Talk on RoboTurk presented at NVIDIA GTC 2021.

Algorithms for Learning Robot Manipulation through Human Imitation

• [January 2021] Talk that covers recent imitation learning algorithms for learning from robot manipulation datasets collected by humans.

Patents

Methods and systems to remotely operate robotic devices (filed)

Inventors: Ajay U. Mandlekar, Yuke Zhu, Animesh Garg, Silvio Savarese, Fei-Fei Li

Media Coverage

RoboTurk: A Crowdsourcing Platform for Imitation Learning in Robotics Tech Xplore, November 21, 2018 “In the future, RoboTurk could become a key resource in the field of robotics, aiding the development of more advanced and better performing robots.”

Robots Learn Tasks from People with Framework Developed by Stanford Researchers Stanford News, October 26, 2018 “With a smartphone and a browser, people worldwide will be able to interact with a robot to speed the process of teaching robots how to do basic tasks.”

Community Service

Workshop Organizer

• Organized Data Generation for Robotics workshop at RSS 2024.

• Organized Overlooked Aspects of Imitation Learning: Systems, Data, Tasks, and Beyond workshop at RSS 2022.

• Co-organized Advances and Challenges in Imitation Learning for Robotics workshop at RSS 2020.

Peer Review

• Reviewer for NeurIPS, ICML, RSS, CoRL, ICRA, IROS, CVPR, IEEE T-RO.