Tengyu Ma
Hi! I am an assistant professor of computer science and statistics at Stanford. My research interests broadly include topics in machine learning and algorithms, such as non-convex optimization, deep learning and its theory, reinforcement learning,
representation learning, distributed optimization, convex relaxation (e.g. sum of squares hierarchy), and high-dimensional statistics.
I received my Ph.D. from the Computer Science Department at Princeton University where I was advised by Professor Sanjeev Arora.
As an undergrad, I studied at Andrew Chi-Chih Yao's CS pilot class at Tsinghua University.
E-mail: firstname + lastname at stanford dot edu
Manuscripts
Colin Wei, Jason D. Lee, Qiang Liu, Tengyu Ma
manuscript
Publications
Yuping Luo*, Huazhe Xu*, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma
to appear in ICLR 2019
Hongyi Zhang, Yann N. Dauphin, Tengyu Ma
to appear in ICLR 2019
with Yu Bai and Andrej Risteski
to appear in ICLR 2019
Yuanzhi Li, Tengyu Ma, and Hongyang Zhang
COLT 2018 (Best Paper Award)
with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski
TACL, 2018
Mikhail Khodak, Nikunj Saunshi, Yingyu Liang, Tengyu Ma, Brandon Stewart, Sanjeev Arora
ACL, 2018
with Moritz Hardt and Benjamin Recht
JLMR, 19(29):1−44, 2018.
Rong Ge, Jason D. Lee, and Tengyu Ma
International Conference on Learning Representations (ICLR) 2017
with Rong Ge
NIPS 2017 (oral). (Best paper in the NIPS 2016 workshop on nonconvex optimization for ML)
with Sanjeev Arora, Rong Ge, Yingyu Liang, and Yi Zhang
ICML, 2017
with Moritz Hardt
International Conference on Learning Representations (ICLR) 2017
with Sanjeev Arora and Yingyu Liang
International Conference on Learning Representations (ICLR) 2017
Jason Lee, Qihang Lin, Tengyu Ma, Tianbao Yang
Journal of Machine Learning Research (JMLR), 18(122):1−43, 2017
Xi Chen, Tengyu Ma, Jiawei Zhang, Yuan Zhou
to appear in Operations research
with Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, and Elad Hazan
STOC 2017
with Sanjeev Arora, Rong Ge, and Andrej Risteski
STOC 2017
with Rong Ge and Jason D. Lee
NIPS (best student paper award), 2016
with Elad Hazan
NIPS 2016
with Jonathan Shi and David Steurer
FOCS 2016
with Sanjeev Arora, Rong Ge, Frederic Koehler, and Ankur Moitra
ICML 2016
with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski
Transactions of the Association for Computational Linguistics (TACL), 4:385-399, 2016
with Mark Braverman, Ankit Garg, Huy L. Nguyen, and David P. Woodruff
STOC 2016
with Avi Wigderson
NIPS 2015
with Sanjeev Arora and Yingyu Liang
ICLR workshop, 2016
with Rong Ge
RANDOM/APPROX 2015
with Dan Garber
and Elad Hazan
ICML 2015
with Sanjeev Arora, Rong Ge, and Ankur Moitra
COLT 2015
with Ankit Garg and Huy Nguyễn
NIPS 2014 (oral)
with Sanjeev Arora, Aditya Bhaskara, and Rong Ge
ICML 2014
with Bo Tang and Yajun Wang
Theory of Computing Systems, 2016
Proceedings of 30th Symposium on Theoretical Aspects of Computer Science (STACS 2013)
Links
blog posts:
Gradient Descent Learns Linear Dynamical Systems
(with Moritz Hardt)
Back-propagation, an introduction (with Sanjeev Arora)
A Framework for Analysing Non-Convex Optimization (with Sanjeev Arora)
Sanjeev's posts regarding our word embedding works
Talks
Recent Progress in the Theory of Deep Learning [slides]
UAI tutorial, Aug 2018, Monterey, USA
Understanding and Improving Generalization in ML and RL
Facebook, Jun 2018, Menlo Park, USA
On the Generalization and Approximability in Generative Adversarial Networks (GANs)
STOC Plenary Session, Jun 2018, LA, USA
Algorithmic Regularization in Over-parameterized Models
STOC workshop, Jun 2018, LA, USA
Learning One-hidden-layer Neural Networks with Landscape Design
Simons Institute, Nov 2017, Berkeley, USA
On the Optimization Landscape of Matrix and Tensor Decomposition Problems
Simons Institute, Sept 2017, Berkeley, USA
Generalization and Equilibrium in Generative Adversarial Nets (GANs)
OpenAI, May 2017, San Francisco, USA
Better Understanding of Non-convex Methods in Machine Learning
Stanford Statistics Department Seminar, Jan 2017, Stanford, USA
MIT EECS Special Seminar, Feb 2017, Cambridge, USA
Stanford CS Seminar, Feb 2017, Stanford, USA
Berkeley CS Seminar, Mar 2017, Berkeley, USA
Columbia CS Seminar, Mar 2017, NYC, USA
Analyzing Non-convex Optimization: Matrix Completion and Linear Residual Networks
MIT algorithms and complexity seminar, Dec 2016, Cambridge, USA
Stanford ML lunch, Nov 2016, Stanford, USA
MSR Talks Series, Nov 2016, Redmond, USA
Matrix Completion has No Spurious Local Minimum
NIPS, Dec 2016, Barcelona, Spain
Columbia theory seminar, Oct 2016, New York City, USA
Yale YNPG seminar, Oct 2016, New Haven, USA
Bekeley, Sept 2016, USA
Sum-of-squares Algorithms for Over-complete Tensor Decomposition
Stanford theory seminar, Sept 2016, Stanford, USA
IAS CSDM seminar, Mar 2016, Princeton, USA
Gradient Descent Learns Linear Dynamical Systems
IMA workshop, May 2016, Minneapolis, USA
Communication Lower Bounds For Statistical Estimation Problems via a Distributed Data Processing Inequality
STOC, Jun 2016, Boston, USA
Invited talk at CISS, Mar 2016, Princeton, USA
The Linear Algebraic Structure of Word Meanings
UW Theory seminar, Apr 2016, Madison, USA
MSR Talk Series, Nov 2015, Redmond, USA
Analyzing Non-convex Optimization for Dictionary Learning
ICML, July 2015, Lille, France
MSR Redmond, Nev 2014, Redmond, USA
Dagstuhl Seminar, Sep 2014, Dagstuhl, Germany
On Communication Cost of Distributed Statistical Estimation and Dimensionality
NIPS, Dec 2014, Montreal, Canada
Provable Bounds for Learning Some Deep Representations
ICML, Jun 2014, Beijing, China
Columbia theory lunch, Feb 2014, NYC, USA
CMU theory lunch, Feb 2014, Pittsburgh, USA
Simulate Greedy Algorithms for Several Submodular Matroid Secretary Problems
30th Symposium on Theoretical Aspects of Computer Science(STACS), Kiel, Germany, Feb 2013
A New Variation of Hat Guessing Games
17th International Computing and Combinatorics Conference(COCOON), Dallas, Texas, Aug 2011
Service
PC committees: ICLR 2018, ITCS 2018, ALT 2018
Journal refereeing: Journal of Machine Learning Research, Mathematics of Operations Research, IEEE Transaction on Information Theory, Optimization Methods and Software, Theoretical Computer Science, Transactions on Pattern Analysis and Machine Intelligence
Conference refereeing: STOC, FOCS, ICML (with outstanding reviewer award in 2016), NIPS, COLT, AAAI(PC member), SODA, ISSAC