The thirty-sixth Conference on Neural Information Processing Systems (NeurIPS) 2022 is being hosted this week. We’re excited to share all the work from SAIL that’s being presented at the main conference, at the Datasets and Benchmarks track and the various workshops, and you’ll find links to papers, videos and blogs below.

Feel free to reach out to the contact authors and the workshop organizers directly to learn more about the work that’s happening at Stanford!

Main Conference

Learning Options via Compression

Authors: Yiding Jiang*, Evan Zheran Liu*, Benjamin Eysenbach, J. Zico Kolter, Chelsea Finn
Contact: evanliu@cs.stanford.edu
Keywords: hierarchical reinforcement learning, skill learning


MABSplit: Faster Forest Training Using Multi-Armed Bandits

Authors: Mo Tiwari, Ryan Kang*, Je-Yong Lee*, Sebastian Thrun, Chris Piech, Ilan Shomorony#, Martin Jinye Zhang#
Contact: Motiwari@stanford.edu
Keywords: multi-armed bandits, random forests


Off-Policy Evaluation for Action-Dependent Non-stationary Environments

Authors: Yash Chandak, Shiv Shankar, Nathaniel D. Bastian, Bruno Castro da Silva, Emma Brunskill, Philip S. Thomas
Contact: ychandak@stanford.edu
Keywords: non-stationarity, off-policy, reinforcement learning, counterfactual


A Fourier Approach to Mixture Learning

Authors: Mingda Qiao, Guru Guruganesh, Ankit Singh Rawat, Avinava Dubey, Manzil Zaheer
Contact: mqiao@stanford.edu
Links: Paper
Keywords: mixture learning, gaussian mixture models


A Nonconvex Framework for Structured Dynamic Covariance Recovery

Authors: Katherine Tsai, Mladen Kolar, Oluwasanmi Koyejo
Contact: tsaikl@stanford.edu
Links: Paper
Keywords: dynamic covariance, structured factor model, alternating projected gradient descent, time series data, functional connectivity


Active Learning Helps Pretrained Models Learn the Intended Task

Authors: Alex Tamkin, Dat Pham Nguyen, Salil Deshpande, Jesse Mu, Noah Goodman
Contact: atamkin@stanford.edu
Links: Paper
Keywords: pretrained models, robustness, active learning, few shot learning


An Information-Theoretic Framework for Deep Learning

Authors: Hong Jun Jeon, Benjamin Van Roy
Contact: hjjeon@stanford.edu
Links: Paper
Keywords: information theory, deep learning, neural network theory


Assistive Teaching of Motor Control Tasks to Humans

Authors: Megha Srivastava, Erdem Biyik, Survir Mirchandani, Noah Goodman, Dorsa Sadigh
Contact: meghas@stanford.edu
Links: Paper | Website
Keywords: human-ai interaction, education, reinforcement learning


Authors: Jeff Z. HaoChen, Colin Wei, Ananya Kumar, Tengyu Ma
Contact: jhaochen@stanford.edu
Links: Paper
Keywords: self-supervised learning theory, deep learning theory


Beyond neural scaling laws: beating power law scaling via data pruning

Authors: Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S Morcos
Contact: bsorsch@gmail.com
Award nominations: Best paper award
Links: Paper
Keywords: scaling laws, deep learning theory, data pruning, replica theory, active learning, data structure


CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior

Authors: Eldar David Abraham, Karel D’Oosterlinck, Amir Feder, Yair Ori Gat, Atticus Geiger, Christopher Potts, Roi Reichart, Zhengxuan Wu
Contact: karel.doosterlinck@ugent.be
Links: Paper | Video | Website


Collaborative Decision Making Using Action Suggestions

Authors: Dylan M. Asmar, Mykel J. Kochenderfer
Contact: asmar@stanford.edu
Links: Paper | Website
Keywords: collaboration, decision making, human-ai collaboration, pomdp, state estimation


Concrete Score Matching: Generalized Score Matching for Discrete Data

Authors: Chenlin Meng*, Kristy Choi*, Jiaming Song, Stefano Ermon
Contact: chenlin@stanford.edu
Links: Paper
Keywords: generative models, score matching, discrete data


Contrastive Adapters for Foundation Model Group Robustness

Authors: Michael Zhang, Christopher Ré
Contact: mzhang@cs.stanford.edu
Links: Paper
Keywords: foundation models, robustness, adaption, efficient


DRAGON: Deep Bidirectional Language-Knowledge Graph Pretraining

Authors: Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang*, Jure Leskovec*
Contact: myasu@cs.stanford.edu
Links: Paper | Blog Post | Website
Keywords: pretraining, language model, knowledge graph, question answering, commonsense, reasoning, foundation model, self-supervised learning, biomedical


Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Data

Authors: Allen Nie, Yannis Flet-Berliac, Deon R. Jordan, William Steenbergen, Emma Brunskill
Contact: anie@stanford.edu
Links: Paper | Website
Keywords: offline rl, hyperparameter selection, data efficient, small data


Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learning

Authors: Dilip Arumugam, Benjamin Van Roy
Contact: dilip@cs.stanford.edu
Links: Paper
Keywords: reinforcement learning, efficient exploration, information theory, bayesian reinforcement learning, value equivalence


Denoising Diffusion Restoration Models

Authors: Bahjat Kawar, Michael Elad, Stefano Ermon, Jiaming Song
Contact: jiaming.tsong@gmail.com
Links: Paper | Website
Keywords: diffusion problems, inverse problems


Distinguishing discrete and continuous behavioral variability using warped autoregressive HMMs

Authors: Julia Costacurta, Lea Duncker, Blue Sheffer, Winthrop Gillis, Caleb Weinreb, Jeffrey Markowitz, Sandeep Datta, Alex Williams, Scott Linderman
Contact: jcostac@stanford.edu
Links: Paper
Keywords: time series, markov models, naturalistic behavior, clustering


ELIGN: Expectation Alignment as a Multi-agent Intrinsic Reward

Authors: Zixian Ma, Rose Wang, Li Fei-Fei, Michael Bernstein, Ranjay Krishna
Contact: zixianma@cs.stanford.edu
Links: Paper
Keywords: multi-agent collaboration, intrinsic reward, reinforcement learning


Estimating and Explaining Model Performance When Both Covariates and Labels Shift

Authors: Lingjiao Chen, Matei Zaharia, James Zou
Contact: lingjiao@stanford.edu
Links: Paper | Website
Keywords: ml models, model deployment and monitoring, data distribution shift


Few-shot Relational Reasoning via Connection Subgraph Pretraining

Authors: Qian Huang, Hongyu Ren, Jure Leskovec
Contact: qhwang@cs.stanford.edu
Links: Paper
Keywords: few-shot learning, knowledge graphs, graph neural networks, self-supervised pretraining


FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Authors: Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré
Contact: trid@stanford.edu
Links: Paper | Video | Website
Keywords: attention, long context, language modeling


Giving Feedback on Interactive Student Programs with Meta-Exploration

Authors: Evan Zheran Liu*, Moritz Stephan*, Allen Nie, Chris Piech, Emma Brunskill, Chelsea Finn
Contact: evanliu@cs.stanford.edu
Award nominations: Selected as oral
Links: Paper | Video | Website
Keywords: meta-reinforcement learning, reinforcement learning, exploration, education


Gradient Estimation with Discrete Stein Operators

Authors: Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis K. Titsias, Lester Mackey
Contact: jiaxins@stanford.edu
Award nominations: Outstanding Paper Award
Links: Paper | Website
Keywords: gradient estimation, stein’s method, discrete latent variables, variance reduction


Improving Intrinsic Exploration with Language Abstractions

Authors: Jesse Mu, Victor Zhong, Roberta Raileanu, Minqi Jiang, Noah Goodman, Tim Rocktäschel, Edward Grefenstette
Contact: muj@stanford.edu
Links: Paper | Video
Keywords: reinforcement learning, intrinsic motivation, exploration, language, deep rl, language-guided rl


Improving Policy Learning via Language Dynamics Distillation

Authors: Victor Zhong, Jesse Mu, Luke Zettlemoyer, Edward Grefenstette, Tim Rocktäschel
Contact: muj@stanford.edu
Links: Paper
Keywords: language grounding, reinforcement learning, reading to generalize


Improving Self-Supervised Learning by Characterizing Idealized Representations

Authors: Yann Dubois, Tatsunori Hashimoto, Stefano Ermon, Percy Liang
Contact: yanndubs@stanford.edu
Links: Paper | Video | Website
Keywords: self-supervised learning, invariances, contrastive learning, machine learning, representation learning


Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces

Authors: Leonard Papenmeier, Luigi Nardi, Matthias Poloczek
Contact: lnardi@stanford.edu
Award nominations: No
Links: Paper | Website
Keywords: high-dimensional global optimization, bayesian optimization, gaussian processes


Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

Authors: Yining Chen, Elan Rosenfeld, Mark Sellke, Tengyu Ma, Andrej Risteski
Contact: cynnjjs@stanford.edu
Links: Paper
Keywords: domain generalization, ood robustness


Joint Entropy Search for Maximally-Informed Bayesian Optimization

Authors: Carl Hvarfner, Frank Hutter, Luigi Nardi
Contact: lnardi@stanford.edu
Award nominations: No
Links: Paper | Website
Keywords: bayesian optimization, entropy search, gaussian processes


Learning to Accelerate Partial Differential Equations via Latent Global Evolution

Authors: Tailin Wu, Takashi Maruyama, Jure Leskovec
Contact: tailin@cs.stanford.edu
Links: Paper | Website
Keywords: accelerate, partial differential equation, latent global evolution, inverse optimization


Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks

Authors: Mansheej Paul*, Brett W. Larsen*, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite
Contact: mansheej@stanford.edu
Links: Paper
Keywords: data pruning, linear mode connectivity, iterative magnitude pruning, loss landscape geometry, lottery ticket hypothesis, sparsity


Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

Authors: Weixin Liang
Contact: wxliang@stanford.edu
Links: Paper | Video | Website
Keywords: cone effect, modality gap, geometry of deep multi-model learning, contrastive representation learning, multi-modal representation learning


No Free Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Entorhinal-Hippocampal Circuit

Authors: Rylan Schaeffer, Mikail Khona, Ila Rani Fiete
Contact: rylanschaeffer@gmail.com
Links: Paper | Blog Post
Keywords: neuroscience, deep learning, grid cells, representation learning


Oracle Inequalities for Model Selection in Offline Reinforcement Learning

Authors: Jonathan N. Lee, George Tucker, Ofir Nachum, Bo Dai, Emma Brunskill
Contact: jnl@stanford.edu
Links: Paper | Website
Keywords: reinforcement learning, offline reinforcement learning, model selection, hyperparameter tuning, offline policy evaluation


Planning to the Information Horizon of BAMDPs via Epistemic State Abstraction

Authors: Dilip Arumugam, Satinder Singh
Contact: dilip@cs.stanford.edu
Links: Paper
Keywords: bayes-adaptive markov decision process, bayesian reinforcement learning, exploration, planning


S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces

Authors: Eric Nguyen*, Karan Goel*, Albert Gu*, Gordon W. Downs, Tri Dao, Preey Shah, Stephen A. Baccus, Christopher Ré
Contact: etnguyen@stanford.edu
Links: Paper | Video | Website
Keywords: state space models, s4, computer vision, deep learning


SIXO: Smoothing Inference with Twisted Objectives

Authors: Dieterich Lawson, Allan Raventos, Andrew Warrington, Scott Linderman
Contact: jdlawson@stanford.edu
Award nominations: Oral
Links: Paper
Keywords: smoothing, variational, objectives, fivo, sequential monte carlo, inference, twisted, time series


STaR: Bootstrapping Reasoning With Reasoning

Authors: Eric Zelikman*, Yuhuai Wu*, Jesse Mu, Noah Goodman
Contact: ezelikman@cs.stanford.edu
Links: Paper
Keywords: chain-of-thought, reasoning, language model, bootstrapping


SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

Authors: Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, Stefano Ermon
Contact: samar.khanna@stanford.edu
Links: Paper | Video | Website
Keywords: self-supervised learning, transformers, pretraining, satellite images, temporal, multi-spectral


Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations

Authors: Michael Poli, Winnie Xu, Stefano Massaroli, Chenlin Meng, Kuno Kim, Stefano Ermon
Contact: poli@stanford.edu
Links: Paper | Website
Keywords: implicit representations, compression, deep equilibrium models, generative models, fractal, fixed-point


Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data

Authors: Armin W. Thomas, Christopher Ré, Russell A. Poldrack
Contact: athms@stanford.edu
Links: Paper
Keywords: self-supervised learning, neuroimaging, deep learning, natural language processing, brain decoding


Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers

Authors: Colin Wei, Yining Chen, Tengyu Ma
Contact: colin.y.wei@gmail.com
Links: Paper
Keywords: approximation theory, generalization bounds, sample complexity bounds, learning theory


Structural Analysis of Branch-and-Cut and the Learnability of Gomory Mixed Integer Cuts

Authors: Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, Ellen Vitercik
Contact: vitercik@stanford.edu
Links: Paper
Keywords: gomory mixed integer cuts, automated algorithm configuration, integer programming, tree search, branch-and-bound, branch-and-cut, cutting planes, sample complexity, generalization guarantees, data-driven algorithm design


Training and Inference on Any-Order Autoregressive Models the Right Way

Authors: Andy Shih, Dorsa Sadigh, Stefano Ermon
Contact: andyshih@stanford.edu
Award nominations: Selected as Oral
Links: Paper | Video | Website
Keywords: any-order autoregressive models, tractable generative models, arbitrary marginal and conditional


Transform Once: Efficient Operator Learning in Frequency Domain

Authors: Michael Poli, Stefano Massaroli, Federico Berto, Jinkyoo Park, Tri Dao, Christopher Re, Stefano Ermon
Contact: poli@stanford.edu
Links: Paper
Keywords: convolutions, long range dependencies, neural operators, high-resolution, frequency, transform, differential equation, dynamics, turbulence, fluid flows, pde


Uncalibrated Models Can Improve Human-AI Collaboration

Authors: Kailas Vodrahalli, Tobias Gerstenberg, James Zou
Contact: kailasv@stanford.edu
Links: Paper
Keywords: human-in-the-loop ai, human-calibrated ai


WeightedSHAP: analyzing and improving Shapley based feature attributions

Authors: Yongchan Kwon, James Zou
Contact: yckwon@stanford.edu
Links: Paper | Website
Keywords: shapley value, model interpretation, attribution problem


What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Authors: Shivam Garg, Dimitris Tsipras, Percy Liang, Gregory Valiant
Contact: shivamg@cs.stanford.edu; tsipras@stanford.edu
Links: Paper
Keywords: in-context learning, transformers, meta-learning


When Does Differentially Private Learning Not Suffer in High Dimensions?

Authors: Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A Inan, Janardhan Kulkarni, YinTat Lee, Abhradeep Guha Thakurta
Contact: lxuechen@cs.stanford.edu
Links: Paper
Keywords: differential privacy, fine-tuning, dp convex optimization, pretrained models


You Only Live Once: Single Life Reinforcement Learning

Authors: Annie S. Chen, Archit Sharma, Sergey Levine, Chelsea Finn
Contact: asc8@stanford.edu
Links: Paper | Website
Keywords: reinforcement learning, autonomous reinforcement learning, adversarial imitation learning


ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time

Authors: Tailin Wu, Megan Tjandrasuwita, Zhengxuan Wu, Xuelin Yang, Kevin Liu, Rok Sosič, Jure Leskovec
Contact: tailin@cs.stanford.edu
Links: Paper | Website
Keywords: zero-shot concept recognition, zero-shot concept acquisition, neuro-symbolic, inference time


5th Robot Learning Workshop: Trustworthy Robotics

Authors: Ransalu Senanayake
Contact: ransalu@stanford.edu
Links: Paper | Website
Keywords: trustworthy ai, robotics

Datasets and Benchmarks Track

Workshop Papers

We look forward to seeing you at NeurIPS 2022!