Awesome Podcast

In this episode we briefly say what we're up to with the podcast.  No technical content, just a description of what each episode will look like, and why we're doing this.

00 - Intro to the podcast

https://www.semanticscholar.org/paper/A-Comparative-Study-of-Word-Embeddings-for-Reading-Dhingra-Liu/3ec37205c9201fc891ab51da200e361fdc34bfb3

01 - A Comparative Study of Word Embeddings for Reading Comprehension

https://www.semanticscholar.org/paper/Bidirectional-Attention-Flow-for-Machine-Seo-Kembhavi/007ab5528b3bd310a80d553cccad4b78dc496b02

02 - Bidirectional Attention Flow for Machine Comprehension

https://www.semanticscholar.org/paper/FastQA-A-Simple-and-Efficient-Neural-Architecture-Weissenborn-Wiese/7c1576b96a1e246d77b30f7b80cec63be96fa698

03 - FastQA: A Simple and Efficient Neural Architecture for Question Answering

An interview with Chris Dyer.

https://www.semanticscholar.org/paper/Recurrent-Neural-Network-Grammars-Dyer-Kuncoro/1594d954abc650bce2db445c52a76e49655efb0c

04 - Recurrent Neural Network Grammars, with Chris Dyer

https://www.semanticscholar.org/paper/A-Syntactic-Neural-Model-for-General-Purpose-Code-Yin-Neubig/c8d0e13de2eaa09a928eff36b99d63f494c2f5ec

10 - A Syntactic Neural Model for General-Purpose Code Generation

12 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

14 - Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning

15 - Attention and Augmented Recurrent Neural Networks

How should you incorporate background knowledge into a neural net? A lot of people have been thinking about this problem, and Dirk Weissenborn comes on to tell us about his work in this area.  Paper is with Tomáš Kočiský and Chris Dyer.

https://arxiv.org/abs/1706.02596

47 - Dynamic integration of background knowledge in neural NLU systems, with Dirk Weißenborn

PUBG (Trap Remix) [Main Theme]

ACL 2018 paper by Amrita Saha, Rahul Aralikatte, Mitesh M. Khapra, Karthik Sankaranarayanan

Amrita and colleagues at IBM Research introduced a harder dataset for "reading comprehension", where you have to answer questions about a given passage of text.  Amrita joins us on the podcast to talk about why a new dataset is necessary, what makes this one unique and interesting, and how well initial baseline systems perform on it.  Along the way, we talk about the problems with using BLEU or ROUGE as evaluation metrics for question answering systems.

https://www.semanticscholar.org/paper/DuoRC%3A-Towards-Complex-Language-Understanding-with-Saha-Aralikatte/1e70a4830840d48486ecfbc6c89b774cdd0b6399

71 - DuoRC: Complex Language Understanding with Paraphrased Reading Comprehension, with Amrita Saha

In this episode, we invite Hal Daumé to continue the discussion on reinforcement learning, focusing on how it has been used in NLP. We discuss how to reduce NLP problems into the reinforcement learning framework, and circumstances where it may or may not be useful. We discuss imitation learning, roll-in and roll-out, and how to approximate an expert with a reference policy. 

DAgger: https://www.semanticscholar.org/paper/A-Reduction-of-Imitation-Learning-and-Structured-to-Ross-Gordon/17eddf33b513ae1134abadab728bdbf6abab2a05?navId=citing-papers
RESLOPE: http://legacydirs.umiacs.umd.edu/~hal/docs/daume18reslope.pdf

75 - Reinforcement / Imitation Learning in NLP, with Hal Daumé III

In this episode, we invite John Hewitt to discuss his take on how to probe word embeddings for syntactic information. The basic idea is to project word embeddings to a vector space where the L2 distance between a pair of words in a sentence approximates the number of hops between them in the dependency tree. The proposed method shows that ELMo and BERT representations, trained with no syntactic supervision, embed many of the unlabeled, undirected dependency attachments between words in the same sentence.

Paper: https://nlp.stanford.edu/pubs/hewitt2019structural.pdf
GitHub repository: https://github.com/john-hewitt/structural-probes
Blog post: https://nlp.stanford.edu/~johnhew/structural-probe.html
Twitter thread: https://twitter.com/johnhewtt/status/1114252302141886464
John's homepage: https://nlp.stanford.edu/~johnhew/

88 - A Structural Probe for Finding Syntax in Word Representations, with John Hewitt

In this episode, we invite Zhou Yu to give an overview of dialogue systems. We discuss different types of dialogue systems (task-oriented vs. non-task-oriented), the main building blocks and how they relate to other research areas in NLP, how to transfer models across domains, and the different ways used to evaluate these systems. Zhou also shares her thoughts on exciting future directions such as developing dialogue methods for non-cooperative environments (e.g., to negotiate prices) and multimodal dialogue systems (e.g., using video as well as audio/text).

Zhou Yu's homepage: http://zhouyu.cs.ucdavis.edu/

89 - Dialog Systems, with Zhou Yu

In this episode, we invite Jonathan Berant to talk about executable semantic parsing. We discuss what executable semantic parsing is and how it differs from related tasks such as semantic dependency parsing and abstract meaning representation (AMR) parsing. We talk about the main components of a semantic parser, how the formal language affects design choices in the parser, and end with a discussion of some exciting open problems in this space.

Jonathan Berant's homepage: http://www.cs.tau.ac.il/~joberant/

91 - (Executable) Semantic Parsing, with Jonathan Berant

In this episode, we invite David Bamman to give an overview of computational humanities. We discuss examples of questions studied in computational humanities (e.g., characterizing fictionality, assessing novelty, measuring the attention given to male vs. female characters in the literature). We talk about the role NLP plays in addressing these questions and how the accuracy and biases of NLP models can influence the results. We also discuss understudied NLP tasks which can help us answer more questions in this domain such as literary scene coreference resolution and constructing a map of literature geography.

David Bamman's homepage: http://people.ischool.berkeley.edu/~dbamman/
LitBank dataset: https://github.com/dbamman/litbank

92 - Computational Humanities, with David Bamman

**The podcast is currently on hiatus. For more active NLP content, check out the Holistic Intelligence Podcast linked below.**

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

47 - Dynamic integration of background knowledge in neural NLU systems, with Dirk Weißenborn

NLP Highlights