Ruohan Zhang

I am a researcher at Stanford Vision and Learning Lab (SVL), as well as a Wu Tsai Human Performance Alliance Fellow. I work on robotics, human-robot interaction, brain-machine interface, neuroscience, and art.

I am currently working with Prof. Fei-Fei Li, Prof. Jiajun Wu, and Prof. Silvio Savarese. I received my Ph.D. from The University of Texas at Austin, advised by Prof. Dana Ballard and Prof. Mary Hayhoe.

Email / CV / Google Scholar / Twitter / Github

Research

My research focus is human-centered artificial intelligence: Understanding human intelligence for developing biologically-inspired AI algorithms, as well as making AIs more compatible with humans.

	Understanding Human Intelligence in the Era of Artificial Intelligence. Ruohan Zhang Research Summary, 2020 paper
	DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, C. Karen Liu arxiv, 2024 paper \| website
	TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation Shivin Dass, Wensi Ai, Yuqian Jiang, Samik Singh, Jiaheng Hu, Ruohan Zhang, Peter Stone, Ben Abbatematteo, Roberto Martín-Martín arxiv, 2024 paper \| website
	NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities Ruohan Zhang, Sharon Lee, Minjune Hwang, Ayano Hiranaka, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu (equally contributed) Bridging the Gap between Cognitive Science and Robot Learning in the Real World Workshop at CoRL, 2023 (Oral)* Conference on Robot Learning (CoRL), 2023 paper \| website \| Stanford HAI News \| MangoTV interview (湖南卫视专访)
	VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Li Fei-Fei Conference on Robot Learning (CoRL), 2023 (Oral) paper \| website
	Mimicplay: Long-horizon Imitation Learning by Watching Human Play Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar Conference on Robot Learning (CoRL), 2023 (Oral, Finalist - Best Paper/Best Student Paper Awards, Finalist - Best Systems Paper Award) paper \| website
	Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI Emily Jin, Jiaheng Hu, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei, Roberto Martín-Martín Agent Learning in Open-Endedness (ALOE) Workshop at NeurIPS, 2023 Generalization in Planning (GenPlan) Workshop at NeurIPS, 2023 paper \| Code
	Quantifying the Effect of Visual Impairments on Daily Activities in Virtual, Interactive Environments Wensi Ai, Sharon Lee, Li Fei-Fei, Jiajun Wu, Ruohan Zhang Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), 2023 paper \| website
	Primitive Skill-based Robot Learning from Human Evaluative Feedback Ayano Hiranaka, Minjune Hwang, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang (equally contributed) International Conference on Intelligent Robots and Systems (IROS)*, 2023 paper \| website
	Partial-View Object View Synthesis via Filtering Inversion Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber International Conference on 3D Vision (3DV) , 2024 paper \| website \| code
	Modeling Dynamic Environments with Scene Graph Memory Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Chengshu Li, Emily Jin, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto Martín-Martín International Conference on Machine Learning (ICML), 2023 paper \| code
	Task-Driven Graph Attention for Hierarchical Relational Object Navigation Michael Lingelbach, Chengshu Li, Minjune Hwang, Andrey Kurenkov, Alan Lou, Roberto Martín-Martín, Ruohan Zhang, Li Fei-Fei, Jiajun Wu International Conference on Robotics and Automation (ICRA), 2023 paper \| code
	BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Wensi Ai, Benjamin Martinez, Hang Yin, Michael Lingelbach, Minjune Hwang, Ayano Hiranaka, Sujay Garlanka, Arman Aydin, Sharon Lee, Jiankai Sun, Mona Anvari, Manasi Sharma, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R Matthews, Ivan Villa-Renteria, Jerry Huayang Tang, Claire Tang, Fei Xia, Yunzhu Li, Silvio Savarese, Hyowon Gweon, C. Karen Liu, Jiajun Wu, Li Fei-Fei (equally contributed) Conference on Robot Learning (CoRL), 2022 (Best Paper Nominee) paper ｜ website
	A Dual Representation Framework for Robot Learning with Human Guidance Ruohan Zhang, Dhruva Bansal, Yilun Hao, Ayano Hiranaka, Jialu Gao, Chen Wang, Roberto Martín-Martín, Li Fei-Fei, Jiajun Wu (equally contributed) Aligning Robot Representations with Humans Workshop at CoRL, 2022 (Spotlight) Conference on Robot Learning (CoRL), 2022 paper \| appendix \| website
	Interaction Modeling with Multiplex Attention Fan-Yun Sun, Isaac Kauvar, Ruohan Zhang, Jiachen Li, Mykel Kochenderfer, Jiajun Wu, Nick Haber Advances in Neural Information Processing Systems (NeurIPS), 2022 link \| code
	How to Train your Decision-Making AIs? Ruohan Zhang, Dhruva Bansal The Gradient, 2022 link \| paper
	Selective Visual Attention during Public Speaking in an Immersive Context Mikael Rubin, Sihang Guo, Karl Muller, Ruohan Zhang, Michael Telch, Mary Hayhoe Attention, Perception, & Psychophysics, 2022 link \| paper \| VSS2021 abstract
	Machine versus Human Attention in Deep Reinforcement Learning Tasks Sihang Guo, Ruohan Zhang, Bo Liu, Yifeng Zhu, Mary Hayhoe, Dana Ballard, Peter Stone Advances in Neural Information Processing Systems (NeurIPS), 2021 link \| paper \| arxiv \| VSS2021 abstract
	Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, Subbarao Kambhampati Advances in Neural Information Processing Systems (NeurIPS), 2021 (Spotlight) link \| paper \| arxiv \| talk
	Recent Advances in Leveraging Human Guidance for Sequential Decision-Making tasks Ruohan Zhang, Faraz Torabi, Garrett Warnell, Peter Stone. (equally contributed) Autonomous Agents and Multi-Agent Systems (JAAMAS)*, 2021 link \| paper \| arxiv
	A Modular Attention Hypothesis for Modeling Visuomotor Behaviors Ruohan Zhang The University of Texas at Austin Ph.D. Dissertation, 2021 paper \| slides
	Efficiently Guiding Imitation Learning Algorithms with Human Gaze Akanksha Saran, Ruohan Zhang, Elaine Schaertl Short, Scott Niekum Autonomous Agents and Multi-Agent Systems (AAMAS), 2021 paper \| arxiv \| bibtex \| code \| slides \| media
	The Hierarchical Evolution in Human Vision Modeling Dana H. Ballard, Ruohan Zhang Topics in Cognitive Sciences, 2021 link \| paper
	Human Gaze Assisted Artificial Intelligence: A Review Ruohan Zhang, Akanksha Saran, Bo Liu, Yifeng Zhu, Sihang Guo, Scott Niekum, Dana Ballard, Mary Hayhoe International Joint Conference on Artificial Intelligence (IJCAI) Survey Track, 2020 paper
	Parallel Neural Processing with Gamma Frequency Latencies Ruohan Zhang, Dana H. Ballard Neural Computation, 2020 link \| paper
	Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset Ruohan Zhang, Calen Walshe, Zhuode Liu, Lin Guan, Karl S. Muller, Jake A. Whritner, Luxin Zhang, Mary M Hayhoe, Dana Ballard AAAI Conference on Artificial Intelligence (AAAI), 2020 link \| paper \| arxiv \| dataset \| code \| poster \| AAAI2020 RLG Workshop talk
	An Initial Attempt of Combining Visual Selective Attention with Deep Reinforcement Learning Liu Yuezhang, Ruohan Zhang, Dana Ballard arxiv, 2020 arxiv
	Leveraging Human Guidance for Deep Reinforcement Learning Tasks Ruohan Zhang, Faraz Torabi, Lin Guan, Dana Ballard, Peter Stone International Joint Conference on Artificial Intelligence (IJCAI) Survey Track, 2019 paper \| arxiv
	AGIL: Learning Attention from Human for Visuomotor Tasks Ruohan Zhang, Zhuode Liu, Luxin Zhang, Jake Whritner, Karl Muller, Mary Hayhoe, Dana Ballard European Conference on Computer Vision (ECCV), 2018 link \| paper \| arxiv \| dataset \| code \| VSS2018 abstract \| VSS2018 talk \| CCN2017 version
	Modeling Sensory-Motor Decisions in Natural Behavior Ruohan Zhang, Shun Zhang, Matthew Tong, Yuchen Cui, Constatin Rothkopf, Dana Ballard, Mary Hayhoe PLoS Computational Biology, 2018 link \| paper \| VSS2017 abstract \| VSS2017 talk \| NETI2016 poster \|
	Model Checking For Safe Navigation Among Humans Sebastian Junges, Nils Jansen, Joost-Pieter Katoen, Ufuk Topcu, Ruohan Zhang, Mary Hayhoe International Conference on Quantitative Evaluation of SysTem (QEST), 2018 link \| paper
	Fast and Precise Black and White Ball Detection for RoboCup Soccer Jacob Menashe, Josh Kelle, Katie Genter, Josiah Hanna, Elad Liebman, Sanmit Narvekar, Ruohan Zhang, Peter Stone RoboCup Symposium, 2017 paper \| code
	Greedy Direction Method of Multiplier for MAP Inference of Large Output Domain Xiangru Huang, Ian E.H. Yen, Ruohan Zhang, Qixing Huang, Pradeep Ravikumar, Inderjit S. Dhillon Artificial Intelligence and Statistics (AISTATS), 2017 paper \| code
	Participatory Art Museum: Collecting and Modeling Crowd Opinions Xiaoyu Zeng, Ruohan Zhang AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2017 link \| paper
	UT Austin Villa: Project-Driven Research in AI and Robotics Katie Genter, Patrick MacAlpine, Jacob Menashe, Josiah Hanna, Elad Liebman, Sanmit Narvekar, Ruohan Zhang, Peter Stone IEEE Intelligent Systems 31(2), 2016 link \| code
	Dual Decomposed Learning with Factorwise Oracle for Structural SVM of Large Output Domain Ian E.H. Yen, Xiangru Huang, Kai Zhong, Ruohan Zhang, Pradeep Ravikumar, Inderjit S. Dhillon Advances in Neural Information Processing Systems (NIPS), 2016 link \| paper
	Decision-Making Policies for Heterogeneous Autonomous Multi-Agent Systems with Safety Constraints Ruohan Zhang, Yue Yu, Mahmoud El Chamie, Behçet Açikmese, and Dana H. Ballard International Joint Conference on Artificial Intelligence (IJCAI), 2016 paper
	Maximum Sustainable Yield Problem for Robot Foraging and Construction System Ruohan Zhang, Zhao Song International Joint Conference on Artificial Intelligence (IJCAI), 2016 paper

Misc.

	Cortical Spikes use Analog Sparse Coding Dana Ballard, Ruohan Zhang, Luc Gentet bioRxiv, 2020 paper
	Attention Guided Imitation Learning and Reinforcement Learning Ruohan Zhang AAAI Conference on Artificial Intelligence (AAAI) Doctoral Consortium, 2019 paper \| NETI2019 poster
	Learning Attention Model From Human for Visuomotor Tasks Luxin Zhang, Ruohan Zhang, Zhuode Liu, Mary Hayhoe, Dana Ballard AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2018 link
	Visual Attention Guided Deep Imitation Learning Ruohan Zhang, Zhuode Liu, Luxin Zhang, Karl Muller, Mary Hayhoe, Dana Ballard. (* equally contributed) NIPS Cognitively Informed Artificial Intelligence Workshop, 2017 paper
	UT Austin Villa 2017 Team Description Paper for the Standard Platform League Katie Genter, Josiah Hanna, Josh Kelle, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Ruohan Zhang, Peter Stone RoboCup Symposium, 2017 paper
	UT Austin Villa 2016 Team Description Paper for the Standard Platform League Katie Genter, Josiah Hanna, Josh Kelle, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Rishi Shah, Ruohan Zhang, Peter Stone RoboCup Symposium, 2016 paper
	UT Austin Villa 2015 Team Description Paper for the Standard Platform League Katie Genter, Josiah Hanna, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Jivko Sinapov, Ruohan Zhang, Peter Stone RoboCup Symposium, 2015 paper
	Global Policy Construction in Modular Reinforcement Learning Ruohan Zhang, Zhao Song, Dana Ballard AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2015 paper

Template from Jon Barron's website.