Tengyu Ma

  Hi! I am an assistant professor of computer science and statistics at Stanford. My research interests broadly include topics in machine learning and algorithms, such as non-convex optimization, deep learning and its theory, reinforcement learning, representation learning, distributed optimization, convex relaxation (e.g. sum of squares hierarchy), and high-dimensional statistics.

I received my Ph.D. from the Computer Science Department at Princeton University where I was advised by Professor Sanjeev Arora. As an undergrad, I studied at Andrew Chi-Chih Yao's CS pilot class at Tsinghua University.
E-mail: firstname + lastname at stanford dot edu
    On the Margin Theory of Feedforward Neural Networks
    Colin Wei, Jason D. Lee, Qiang Liu, Tengyu Ma
    Algorithmic Framework for Model-based Reinforcement Learning with Theoretical Guarantees
    Yuping Luo*, Huazhe Xu*, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma
    to appear in ICLR 2019 
    Hongyi Zhang, Yann N. Dauphin, Tengyu Ma
    to appear in ICLR 2019 
    with Yu Bai and Andrej Risteski
    to appear in ICLR 2019  
    Yuanzhi Li, Tengyu Ma, and Hongyang Zhang
    COLT 2018 (Best Paper Award)  
    with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski
    TACL, 2018  
    Mikhail Khodak, Nikunj Saunshi, Yingyu Liang, Tengyu Ma, Brandon Stewart, Sanjeev Arora
    ACL, 2018  
    with Moritz Hardt and Benjamin Recht
    JLMR, 19(29):1−44, 2018.  
    Rong Ge, Jason D. Lee, and Tengyu Ma
    International Conference on Learning Representations (ICLR) 2017 
    with Rong Ge
    NIPS 2017 (oral). (Best paper in the NIPS 2016 workshop on nonconvex optimization for ML) 
    with Sanjeev Arora, Rong Ge, Yingyu Liang, and Yi Zhang
    ICML, 2017  
    with Moritz Hardt
    International Conference on Learning Representations (ICLR) 2017  
    with Sanjeev Arora and Yingyu Liang
    International Conference on Learning Representations (ICLR) 2017  
    Jason Lee, Qihang Lin, Tengyu Ma, Tianbao Yang
    Journal of Machine Learning Research (JMLR), 18(122):1−43, 2017  
    Xi Chen, Tengyu Ma, Jiawei Zhang, Yuan Zhou
    to appear in Operations research  
    with Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, and Elad Hazan
    STOC 2017  
    with Sanjeev Arora, Rong Ge, and Andrej Risteski
    STOC 2017  
    with Rong Ge and Jason D. Lee
    NIPS (best student paper award), 2016  
    with Elad Hazan
    NIPS 2016  
    with Jonathan Shi and David Steurer
    FOCS 2016  
    with Sanjeev Arora, Rong Ge, Frederic Koehler, and Ankur Moitra
    ICML 2016  
    with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski
    Transactions of the Association for Computational Linguistics (TACL), 4:385-399, 2016  
    with Mark Braverman, Ankit Garg, Huy L. Nguyen, and David P. Woodruff
    STOC 2016  
    with Avi Wigderson
    NIPS 2015  
    with Sanjeev Arora and Yingyu Liang
    ICLR workshop, 2016  
    with Rong Ge
    RANDOM/APPROX 2015  
    with Dan Garber and Elad Hazan
    ICML 2015  
    with Sanjeev Arora, Rong Ge, and Ankur Moitra
    COLT 2015  
    with Ankit Garg and Huy Nguyễn
    NIPS 2014 (oral)  
    with Sanjeev Arora, Aditya Bhaskara, and Rong Ge
    ICML 2014 
    with Bo Tang and Yajun Wang
    Theory of Computing Systems, 2016  
    Proceedings of 30th Symposium on Theoretical Aspects of Computer Science (STACS 2013) 
blog posts:
    Back-propagation, an introduction (with Sanjeev Arora)
Sanjeev's posts regarding our word embedding works
    Recent Progress in the Theory of Deep Learning [slides]
    UAI tutorial, Aug 2018, Monterey, USA
    Understanding and Improving Generalization in ML and RL
    Facebook, Jun 2018, Menlo Park, USA
    On the Generalization and Approximability in Generative Adversarial Networks (GANs)
    STOC Plenary Session, Jun 2018, LA, USA
    Algorithmic Regularization in Over-parameterized Models
    STOC workshop, Jun 2018, LA, USA
    Simons Institute, Jun 2018, Berkeley, USA
    University of Chicago Statistics Colloquium, May 2018, Chicago, USA
    Wharton Statistics Seminar, Mar 2018, Philadelphia, USA
    Conference on Information Sciences and Systems (CISS), Mar 2018, Princeton, USA
    Berkeley Neyman Seminar, Feb 2018, Berkeley, USA
    UC Davis Statistics Seminar, Feb 2017, Davis, USA
    Google, Feb 2017, Mountain View, USA
    Learning One-hidden-layer Neural Networks with Landscape Design
    Simons Institute, Nov 2017, Berkeley, USA
    On the Optimization Landscape of Matrix and Tensor Decomposition Problems
    Simons Institute, Sept 2017, Berkeley, USA
    NIPS, Dec 2017, Long Beach, USA
    Generalization and Equilibrium in Generative Adversarial Nets (GANs)
    OpenAI, May 2017, San Francisco, USA
    Better Understanding of Non-convex Methods in Machine Learning
    Stanford Statistics Department Seminar, Jan 2017, Stanford, USA
    MIT EECS Special Seminar, Feb 2017, Cambridge, USA
    Stanford CS Seminar, Feb 2017, Stanford, USA
    Berkeley CS Seminar, Mar 2017, Berkeley, USA
    Columbia CS Seminar, Mar 2017, NYC, USA
    CMU CSC Seminar, Mar 2017, Pittsburgh, USA
    Caltech CMS Special Seminar, Mar 2017, Pasadena, USA
    UW Computer Science Engineering Colloquium, April 2017, Seattle, USA
    OpenAI, April 2017, San Francisco, USA
    Facebook AI Research, May 2017, Menlo Park, USA
    Analyzing Non-convex Optimization: Matrix Completion and Linear Residual Networks
    MIT algorithms and complexity seminar, Dec 2016, Cambridge, USA
    Stanford ML lunch, Nov 2016, Stanford, USA
    MSR Talks Series, Nov 2016, Redmond, USA
    Matrix Completion has No Spurious Local Minimum
    NIPS, Dec 2016, Barcelona, Spain
    Columbia theory seminar, Oct 2016, New York City, USA
    Yale YNPG seminar, Oct 2016, New Haven, USA
    Bekeley, Sept 2016, USA
    Sum-of-squares Algorithms for Over-complete Tensor Decomposition
    Stanford theory seminar, Sept 2016, Stanford, USA
    IAS CSDM seminar, Mar 2016, Princeton, USA
    Gradient Descent Learns Linear Dynamical Systems
    IMA workshop, May 2016, Minneapolis, USA
    Communication Lower Bounds For Statistical Estimation Problems via a Distributed Data Processing Inequality
    STOC, Jun 2016, Boston, USA
    Invited talk at CISS, Mar 2016, Princeton, USA
    The Linear Algebraic Structure of Word Meanings
    UW Theory seminar, Apr 2016, Madison, USA
    MSR Talk Series, Nov 2015, Redmond, USA
    Analyzing Non-convex Optimization for Dictionary Learning
    ICML, July 2015, Lille, France
    MSR Redmond, Nev 2014, Redmond, USA
    Dagstuhl Seminar, Sep 2014, Dagstuhl, Germany
    On Communication Cost of Distributed Statistical Estimation and Dimensionality
    NIPS, Dec 2014, Montreal, Canada
    Provable Bounds for Learning Some Deep Representations
    ICML, Jun 2014, Beijing, China
    Columbia theory lunch, Feb 2014, NYC, USA
    CMU theory lunch, Feb 2014, Pittsburgh, USA
    Simulate Greedy Algorithms for Several Submodular Matroid Secretary Problems
    30th Symposium on Theoretical Aspects of Computer Science(STACS), Kiel, Germany, Feb 2013
    A New Variation of Hat Guessing Games
    17th International Computing and Combinatorics Conference(COCOON), Dallas, Texas, Aug 2011
    PC committees: ICLR 2018, ITCS 2018, ALT 2018
    Journal refereeing: Journal of Machine Learning Research, Mathematics of Operations Research, IEEE Transaction on Information Theory, Optimization Methods and Software, Theoretical Computer Science, Transactions on Pattern Analysis and Machine Intelligence
    Conference refereeing: STOC, FOCS, ICML (with outstanding reviewer award in 2016), NIPS, COLT, AAAI(PC member), SODA, ISSAC