Yuke Zhu

I am a Ph.D. student at Stanford University. My goal is to build intelligence for general-purpose robots that understand and interact with the visual world.

My research lies at the intersection of robotics, computer vision, and machine learning. I focus on developing methods and mechanisms of perception and control for general-purpose robots. I work in Stanford Vision and Learning Lab with Prof. Fei-Fei Li and Prof. Silvio Savarese. I am also a member of Stanford People, AI & Robots Group. Prior to coming to Stanford, I received a BSc. degree from Simon Fraser University and a BEng. degree from Zhejiang University.

Email: yukez@cs.stanford.edu

Gates Computer Science Building, Room 242
353 Serra Mall, Stanford University
Stanford, CA 94305-9025, USA


[new] Two papers accepted in IROS 2019 and one paper accepted in ICCV 2019.

[new] We won the Best Conference Paper Award at ICRA 2019 in Montreal.

Our paper on multimodal representaiton learning is accepted in ICRA 2019.

We released our new work on 6D pose estimation. Code is available on GitHub.

We released our code and data for Surreal and RoboTurk, which are covered by a Stanford News article.

Selected Publications

  • Situational Fusion of Visual Representation for Visual Navigation
    William Shen, Danfei Xu, Yuke Zhu, Li Fei-Fei, Leonidas Guibas, Silvio Savarese
    ICCV 2019
  • Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset
    through Human Reasoning and Dexterity
    Ajay Mandlekar, Jonathan Booher, Max Spero, Albert Tung, Anchit Gupta, Yuke Zhu, Animesh Garg,
    Silvio Savarese, Li Fei-Fei
    IROS 2019
  • DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
    Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese
    CVPR 2019
  • Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
    De-An Huang*, Suraj Nair*, Danfei Xu*, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese,
    Juan Carlos Niebles
    CVPR 2019
  • Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for
    Contact-Rich Tasks
    Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg,
    Jeannette Bohg
    ICRA 2019 (Best Conference Paper)
  • SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark
    Linxi Fan*, Yuke Zhu*, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese,
    Li Fei-Fei
    CoRL 2018
  • RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
    Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao,
    John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei
    CoRL 2018
  • Reinforcement and Imitation Learning for Diverse Visuomotor Skills
    Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool,
    János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess
    RSS 2018
  • Learning Task-Oriented Grasping for Tool Manipulation with Simulated Self-Supervision
    Kuan Fang, Yuke Zhu, Animesh Garg, Virja Mehta, Andrey Kuryenkov, Li Fei-Fei, Silvio Savarese
    RSS 2018
  • Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
    Danfei Xu*, Suraj Nair*, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
    ICRA 2018
  • ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems
    James Harrison*, Animesh Garg*, Boris Ivanovic, Yuke Zhu, Silvio Savarese, Li Fei-Fei, Marco Pavone
    ISRR 2017
  • Visual Semantic Planning using Deep Successor Representations
    Yuke Zhu*, Daniel Gordon*, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi
    ICCV 2017
  • Scene Graph Generation by Iterative Message Passing
    Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
    CVPR 2017
  • Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
    Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, Ali Farhadi
    ICRA 2017
  • Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen,
    Yannis Kalanditis, Li-Jia Li, David A. Shamma, Michael Bernstein, Li Fei-Fei
    IJCV 2017
  • Visual7W: Grounded Question Answering in Images
    Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei
    CVPR 2016
  • Reasoning About Object Affordances in a Knowledge Base Representation
    Yuke Zhu, Alireza Fathi, Li Fei-Fei
    ECCV 2014

Teaching Experience

  • Teaching Assistant

    Spring 2013-2014 | Stanford, CA, USA

    CS 431: High-Level Vision: Behaviors, Neurons and Computational Models

    Summer 2013-2014 | Stanford, CA, USA

    CS 193C: Client-Side Internet Technologies

    Fall 2014-2015 | Stanford, CA, USA

    CS 131: Computer Vision: Foundations and Applications

    Winter 2014-2015 | Stanford, CA, USA

    CS 231N: Convolutional Neural Networks for Visual Recognition

Working Experience

  • Research Intern

    Jun - Sept 2017 | London, England, United Kingdom

  • Research Intern

    Jun - Sept 2016 | Seattle, WA, USA

    Allen Institute for Artificial Intelligence
  • Research Intern

    May - Aug 2015 | Venice, CA, USA

    Snap Inc.
  • Software Engineer Intern

    Apr - Jul 2013 | San Francisco, CA, USA

    Twitter Inc.
  • Research Assistant

    Dec 2011 - Apr 2013 | Vancouver, BC, Canada

    SFU Computational Logic Lab

    Jan 2012 - Apr 2013 | Vancouver, BC, Canada

    SFU Vision and Media Lab