Hi! I am currently a visiting scientist at Facebook AI Research, located in Menlo Park. My research interests broadly include topics in machine learning and algorithms, such as non-convex optimization, deep learning, representation learning (for natural language processing), distributed optimization, convex relaxation (e.g. sum of squares hierarchy), and high-dimensional statistics. I received my Ph.D. from the Computer Science Department at Princeton University where I was advised by Professor Sanjeev Arora. Before coming to U.S., I studied at Andrew Chi-Chih Yao's CS pilot class at Tsinghua University.

I'm thrilled to be joining Stanford University as an assistant professor in the Fall of 2018! (I'm also currently visiting Stanford every Wed.)

E-mail: firstname + lastname at stanford dot edu

Manuscripts

Linear Algebraic Structure of Word Senses, with Applications to Polysemy

with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski

manuscript, 2016

Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

Yuanzhi Li, Tengyu Ma, and Hongyang Zhang

manuscript, 2018

Gradient Descent Learns Linear Dynamical Systems

with Moritz Hardt and Benjamin Recht

manuscript (submitted to JMLR)

Optimal Design of Process Flexility for General Production Systems

Xi Chen, Tengyu Ma, Jiawei Zhang, Yuan Zhou

manuscript (submitted to Operations research)

Publications

Learning One-hidden-layer Neural Networks with Landscape Design

Rong Ge, Jason D. Lee, and Tengyu Ma

International Conference on Learning Representations (ICLR) 2017

On the Optimization Landscape of Tensor decompositions

with Rong Ge

NIPS 2017 (oral). (Best paper in the NIPS 2016 workshop on nonconvex optimization for ML)

Generalization and Equilibrium in Generative Adversarial Nets (GANs)

with Sanjeev Arora, Rong Ge, Yingyu Liang, and Yi Zhang

ICML, 2017

Identity Matters in Deep Learning

with Moritz Hardt

International Conference on Learning Representations (ICLR) 2017

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

with Sanjeev Arora and Yingyu Liang

International Conference on Learning Representations (ICLR) 2017

Distributed Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement

Jason Lee, Qihang Lin, Tengyu Ma, Tianbao Yang

Journal of Machine Learning Research (JMLR), 18(122):1−43, 2017

Finding Approximate Local Minima for Nonconvex Optimization in Linear Time

with Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, and Elad Hazan

STOC 2017

Provable learning of Noisy-or Networks

with Sanjeev Arora, Rong Ge, and Andrej Risteski

STOC 2017

Matrix Completion has No Spurious Local Minimum

with Rong Ge and Jason D. Lee

NIPS (best student paper award), 2016

A Non-generative Framework and Convex Relaxations for Unsupervised Learning

with Elad Hazan

NIPS 2016

Polynomial-time Tensor Decompositions with Sum-of-squares

with Jonathan Shi and David Steurer

FOCS 2016

Provable Algorithms for Inference in Topic Models

with Sanjeev Arora, Rong Ge, Frederic Koehler, and Ankur Moitra

ICML 2016

RAND-WALK: A Latent Variable Model Approach to Word Embeddings

with Sanjeev Arora, Yuanzhi Li, Yingyu Liang, and Andrej Risteski

Transactions of the Association for Computational Linguistics (TACL), 4:385-399, 2016

Communication Lower Bounds for Statistical Estimation Problems via a Distributed Data Processing Inequality

with Mark Braverman, Ankit Garg, Huy L. Nguyen, and David P. Woodruff

STOC 2016

Sum-of-Squares Lower Bounds for Sparse PCA

with Avi Wigderson

NIPS 2015

Why are deep nets reversible: A simple theory, with implications for training

with Sanjeev Arora and Yingyu Liang

ICLR workshop, 2016

Decomposing Overcomplete 3rd Order Tensors using Sum-of-Squares Algorithms

with Rong Ge

RANDOM/APPROX 2015

Online Learning of Eigenvectors

with Dan Garber and Elad Hazan

ICML 2015

Simple, Efficient, and Neural Algorithms for Sparse Coding

with Sanjeev Arora, Rong Ge, and Ankur Moitra

COLT 2015

On the Communication Cost of Distributed Statistical Estimation and Dimensionality

with Ankit Garg and Huy Nguyễn

NIPS 2014 (oral)

Provable Bounds for Learning Some Deep Representations

with Sanjeev Arora, Aditya Bhaskara, and Rong Ge

ICML 2014

Simulated Greedy Algorithms for Several Submodular Matroid Secretary Problems

with Bo Tang and Yajun Wang

Theory of Computing Systems, 2016

Proceedings of 30th Symposium on Theoretical Aspects of Computer Science (STACS 2013)

Links

blog posts:

Gradient Descent Learns Linear Dynamical Systems (with Moritz Hardt)

Back-propagation, an introduction (with Sanjeev Arora)

A Framework for Analysing Non-Convex Optimization (with Sanjeev Arora)

Sanjeev's posts regarding our word embedding works

Linear algebraic structure of word meanings

Word Embeddings: Explaining their properties

Talks

Learning One-hidden-layer Neural Networks with Landscape Design

Simons Institute, Nov 2017, Berkeley, USA

On the Optimization Landscape of Matrix and Tensor Decomposition Problems

Simons Institute, Sept 2017, Berkeley, USA

NIPS, Dec 2017, Long Beach, USA

Generalization and Equilibrium in Generative Adversarial Nets (GANs)

OpenAI, May 2017, San Francisco, USA

Better Understanding of Non-convex Methods in Machine Learning

Stanford Statistics Department Seminar, Jan 2017, Stanford, USA

MIT EECS Special Seminar, Feb 2017, Cambridge, USA

Stanford CS Seminar, Feb 2017, Stanford, USA

Berkeley CS Seminar, Mar 2017, Berkeley, USA

Columbia CS Seminar, Mar 2017, NYC, USA

CMU CSC Seminar, Mar 2017, Pittsburgh, USA

Caltech CMS Special Seminar, Mar 2017, Pasadena, USA

UW Computer Science Engineering Colloquium, April 2017, Seattle, USA

OpenAI, April 2017, San Francisco, USA

Facebook AI Research, May 2017, Menlo Park, USA

Analyzing Non-convex Optimization: Matrix Completion and Linear Residual Networks

MIT algorithms and complexity seminar, Dec 2016, Cambridge, USA

Stanford ML lunch, Nov 2016, Stanford, USA

MSR Talks Series, Nov 2016, Redmond, USA

Matrix Completion has No Spurious Local Minimum

NIPS, Dec 2016, Barcelona, Spain

Columbia theory seminar, Oct 2016, New York City, USA

Yale YNPG seminar, Oct 2016, New Haven, USA

Bekeley, Sept 2016, USA

Sum-of-squares Algorithms for Over-complete Tensor Decomposition

Stanford theory seminar, Sept 2016, Stanford, USA

IAS CSDM seminar, Mar 2016, Princeton, USA

Gradient Descent Learns Linear Dynamical Systems

IMA workshop, May 2016, Minneapolis, USA

Communication Lower Bounds For Statistical Estimation Problems via a Distributed Data Processing Inequality

STOC, Jun 2016, Boston, USA

Invited talk at CISS, Mar 2016, Princeton, USA

The Linear Algebraic Structure of Word Meanings

UW Theory seminar, Apr 2016, Madison, USA

MSR Talk Series, Nov 2015, Redmond, USA

Analyzing Non-convex Optimization for Dictionary Learning

ICML, July 2015, Lille, France

MSR Redmond, Nev 2014, Redmond, USA

Dagstuhl Seminar, Sep 2014, Dagstuhl, Germany

On Communication Cost of Distributed Statistical Estimation and Dimensionality

NIPS, Dec 2014, Montreal, Canada

Provable Bounds for Learning Some Deep Representations

ICML, Jun 2014, Beijing, China

Columbia theory lunch, Feb 2014, NYC, USA

CMU theory lunch, Feb 2014, Pittsburgh, USA

Simulate Greedy Algorithms for Several Submodular Matroid Secretary Problems

30th Symposium on Theoretical Aspects of Computer Science(STACS), Kiel, Germany, Feb 2013

A New Variation of Hat Guessing Games

17th International Computing and Combinatorics Conference(COCOON), Dallas, Texas, Aug 2011

Service

PC committees: ITCS 2018, ALT 2018

Journal refereeing: Journal of Machine Learning Research, Mathematics of Operations Research, IEEE Transaction on Information Theory, Optimization Methods and Software, Theoretical Computer Science, Transactions on Pattern Analysis and Machine Intelligence

Conference refereeing: STOC, FOCS, ICML (with outstanding reviewer award in 2016), NIPS, COLT, AAAI(PC member), SODA, ISSAC