Page 2 of 3 for NLP Posts - Page 2

Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious

In this post, we examine in-context learning in GPT-3 through a series of experiments dissecting GPT-3's ability to extrapolate to less natural inputs.

An Introduction to Knowledge Graphs

Vinay K. Chaudhri, Naren Chittar, Michael Genesereth

What are Knowledge Graphs and how are they useful in search, data integration and AI?

Inside Chirpy Cardinal: Stanford's Open-Source Social Chatbot that Won 2nd place in the Alexa Prize

Ashwin Paranjape and Amelia Hardy

Last year, Stanford won 2nd place in the Alexa Prize Socialbot Grand Challenge for social chatbots. In this post, we look into building a chatbot that combines the flexibility and naturalness of neural dialog generation with the reliability and practicality of scripted dialogue.

Do Language Models Know How Heavy an Elephant Is?

Xikun Zhang

Humans have a pretty good sense of scale, like the weight or price of different objects, but do pre-trained language representations? We measure how well pre-trained text representations capture scale and also come up with a improved version of BERT that captures scale better.

Learning from Language Explanations

Jesse Mu and Shikhar Murty

Language is a crucial way for humans to teach other humans. Can we use language to teach machines?

Bootleg: Chasing the Tail with Self-Supervised Named Entity Disambiguation

Laurel Orr, Megan Leszczynski, Simran Arora, Neel Guha, Xiao Ling, Sen Wu, and Chris Ré

Identifying rare people, places, and things (entities) in text is a critical problem in AI, especially as the majority of entities are rare. We address this problem in our self-supervised system Bootleg. Bootleg learns to reason over entity types and relations for improved rare entity performance.

Measuring Bias in NLP (with Confidence!)

Kawin Ethayarajh

When estimating bias in NLP, there is usually uncertainty. We propose capturing this uncertainty using confidence intervals, finding that the datasets currently used to measure bias are too small to conclusively identify it – except in the most egregious cases.

Learning to Fix Programs from Error Messages

Michihiro Yasunaga

We study how to use machine learning to repair programs from error messages, and introduce a promising approach that leverages program-feedback graphs and self-supervised learning.

How to Fill in the Blanks with Language Models

Mina Lee

We present a simple way to extend the capabilities of language models to fill in the blanks. Our framework constructs training examples by masking random spans to generate pairs of inputs and targets and fine-tunes language models on the concatenation of each pair.

Finding Cross-Lingual Syntax in Multilingual BERT

Ethan A. Chi

Large neural language models such as BERT have seen surprising success with multilingual pre-training on a large number of languages. We demonstrate that Multilingual BERT learns cross-lingual syntax, visualizing its inherent structure.