NLP Playlist

nlp highlights

deasd

Wouldn’t it solve so many problems in the world if everyone just spoke the same language? Not so fast! 

Lingthusiasm is a brand-new podcast that’s enthusiastic about linguistics, hosted by Lauren Gawne of Superlinguo​ and Gretchen McCulloch of All Things Linguistic.

In this first episode of Lingthusiasm, ​Gretchen and Lauren discuss the “one language equals peace” fallacy, and whether speaking the same words means that people will necessarily agree with each other (spoiler: no). But the history of how people have tried is still really interesting, from constructed and symbolic communication like Blissymbols and emoji to the way astronauts communicate in the high stakes environment of the International Space Station.

For links to things mentioned during this episode visit http://lingthusiasm.com/post/154520059101/lingthusiasm-episode-1-speaking-a-single-language

Listen to bonus episodes, suggest future topics, and help keep the show ad-free by supporting us on Patreon: http://patreon.com/lingthusiasm

01: Speaking a single language won’t bring about world peace

There are some pretty funny quotes of historical people complaining about kids back then doing linguistic things that now seem totally unremarkable. So let’s cut to the chase and celebrate linguistic innovation while it’s happening. 

In episode 7 of Lingthusiasm, your hosts Lauren Gawne and Gretchen McCulloch explore how far back we can trace complaints about the language of Kids These Days, why linguistic discrimination is harmful, and why “be like”, hyperbolic “literally”, and other modern innovations are actually signs of something awesome. 

We also announce a Patreon to keep the podcast sustainable! You can support us there to listen to a bonus episode about swearing and future monthly bonus content, and help decide the topics of future episodes. Even if you’re not sure about pledging, do check out the Patreon goals to see some of our future plans and our video to see a totally realistic and not at all staged cameo from our producer. Check out http://patreon.com/lingthusiasm 

For more information visit the show page: http://lingthusiasm.com/post/159796192161/lingthusiasm-episode-7-kids-these-days-arent

07: Kids these days aren’t ruining language

https://www.semanticscholar.org/paper/A-Comparative-Study-of-Word-Embeddings-for-Reading-Dhingra-Liu/3ec37205c9201fc891ab51da200e361fdc34bfb3

01 - A Comparative Study of Word Embeddings for Reading Comprehension

https://www.semanticscholar.org/paper/Bidirectional-Attention-Flow-for-Machine-Seo-Kembhavi/007ab5528b3bd310a80d553cccad4b78dc496b02

02 - Bidirectional Attention Flow for Machine Comprehension

https://www.semanticscholar.org/paper/FastQA-A-Simple-and-Efficient-Neural-Architecture-Weissenborn-Wiese/7c1576b96a1e246d77b30f7b80cec63be96fa698

03 - FastQA: A Simple and Efficient Neural Architecture for Question Answering

An interview with Chris Dyer.

https://www.semanticscholar.org/paper/Recurrent-Neural-Network-Grammars-Dyer-Kuncoro/1594d954abc650bce2db445c52a76e49655efb0c

04 - Recurrent Neural Network Grammars, with Chris Dyer

https://www.semanticscholar.org/paper/Transition-Based-Dependency-Parsing-with-Stack-Lon-Dyer-Ballesteros/396b7932beac62a72288eaea047981cc9a21379a

05 - Transition-Based Dependency Parsing with Stack Long Short-Term Memory

https://www.semanticscholar.org/paper/Design-Challenges-for-Entity-Linking-Ling-Singh/aa2a7ac7bfa9a0201d4faddd4e7bb26299a5e0be

06 - Design Challenges for Entity Linking

https://www.semanticscholar.org/paper/Capturing-Semantic-Similarity-for-Entity-Linking-w-Francis-Landau-Durrett/1c9aca60f7ac5edcceb73d612806704a7d662643

07 - Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

How do we get from knowing words to making brand-new sentences out of them? In episode 9 of Lingthusiasm, your hosts Gretchen McCulloch and Lauren Gawne talk about how words form groups with other words: constituency. 

Once you start looking for it, constituency is everywhere: in ambiguous sentences like “time flies like an arrow; fruit flies like a banana”, in remixed films like “Of Oz The Wizard”, and even internet dog memes.

This month’s Patreon bonus was the backstory about the linguistics of the doggo meme and its connection to Australian slang, which grew out of this NPR article about doggo. You can get access to it and previous bonuses about swearing, teaching yourself linguistics, and explaining linguistics to employers by supporting Lingthusiasm on Patreon.

For more information, and links to things mentioned in this episode, visit the show page: http://lingthusiasm.com/post/161859273886/lingthusiasm-episode-9-the-bridge-between-words

09: The bridge between words and sentences - Constituency

An interview with Noah Smith.

Noah tells us about his work with his students Hao Peng and Sam Thomson.  We talk about what semantic dependency parsing is, the model that they used to approach the problem, how multi-task learning fits into this with a graph-based parser, and end with a little discussion about representation learning.

https://www.semanticscholar.org/paper/Deep-Multitask-Learning-for-Semantic-Dependency-Pa-Peng-Thomson/406fd41b360bb02c0aaabff54055193fb5d9d7f1

22 -  Deep Multitask Learning for Semantic Dependency Parsing, with Noah Smith

ICLR 2017 paper, by Yoon Kim, Carl Denton, Luong Hoang, and Sasha Rush.

Yoon comes on to talk with us about his paper.  The paper shows how standard attentions can be seen as an expected feature count computation, and can be generalized to other kinds of expected feature counts, as long as we have efficient, differentiable algorithms for computing those marginals, like the forward-backward and inside-outside algorithms.  We talk with Yoon about how this works, the experiments they ran to test this idea, and interesting implications of their work.

https://www.semanticscholar.org/paper/Structured-Attention-Networks-Kim-Denton/0aec1745d0e054e8d86d21b20d0ee5fc0d932a49

Yoon also brought up a more recent paper by Yang Liu and Mirella Lapata that computes a very similar kind of structured attention, but does so much more efficiently.  That paper is here: https://www.semanticscholar.org/paper/Learning-Structured-Text-Representations-Liu-Lapata/4435c3586364e8f8a2c8c9ee671c39d7df7e196c.

26 - Structured Attention Networks, with Yoon Kim

NIPS 2016 paper by Alexander Ratner and coauthors in Chris Ré's group at Stanford, presented by Waleed.

The paper presents a method for generating labels for an unlabeled dataset by combining a number of weak labelers.  This changes the annotation effort from looking at individual examples to constructing a large number of noisy labeling heuristics, a task the authors call "data programming".  Then you learn a model that intelligently aggregates information from the weak labelers to create a weighted "supervised" training set.  We talk about this method, how it works, how it's related to ideas like co-training, and when you might want to use it.

https://www.semanticscholar.org/paper/Data-Programming-Creating-Large-Training-Sets-Quic-Ratner-Sa/37acbbbcfe9d8eb89e5b01da28dac6d44c3903ee

28 - Data Programming: Creating Large Training Sets, Quickly

ACL 2017 paper, by Yusuke Oda and others (including Graham Neubig) at Nara Institute of Science and Technology (Graham is now at Carnegie Mellon University).

Graham comes on to talk to us about neural machine translation generally, and about this ACL paper in particular.  We spend the first half of the episode talking about major milestones in neural machine translation and why it is so much more effective than previous methods (spoiler: stronger language models help a lot).  We then talk about the specifics of binary code prediction, how it's related to a hierarchical or class-factored softmax, and how to make it robust to off-by-one-bit errors.

Paper link: https://www.semanticscholar.org/paper/Neural-Machine-Translation-via-Binary-Code-Predict-Oda-Arthur/bbedfd0380eb2e62f1c3b61aaf484d5867e6358d

An example of the Language log posts that we discussed: http://languagelog.ldc.upenn.edu/nll/?p=33613 (there are many more).

29 - Neural machine translation via binary code prediction, with Graham Neubig

Some linguists work with multiple languages, while others focus on just one. But for many people, learning a language after early childhood is the thing that first gets them curious about how language works in general and all the things in their native language(s) that they take for granted. 

In episode 10 of Lingthusiasm, your hosts Gretchen McCulloch and Lauren Gawne talk about how learning languages can feed into linguistics and vice versa. We also explore the power dynamics that affect learning languages, and the importance of learning about the rules of interaction as well as the rules of grammar.

This month’s Patreon bonus was about hypercorrection, where you try so hard to follow a linguistic rule that you end up overshooting. You can get access to it and previous bonuses about the doggo meme, swearing, teaching yourself linguistics, and explaining linguistics to employers by supporting Lingthusiasm on Patreon. http://patreon.com/lingthusiasm

Gretchen’s new recorder in this episode is thanks to the support of our patrons! 

For more information, and links to things mentioned in this episode, visit the show page: http://lingthusiasm.com/post/163226798571/lingthusiasm-episode-10-learning-languages

10: Learning languages linguistically

NAACL 2018 paper, by Rachel Rudinger, Aaron Steven White, and Benjamin Van Durme

Rachel comes on to the podcast, telling us about what factuality is (did an event happen?), what datasets exist for doing this task (a few; they made a new, bigger one), and how to build models to predict factuality (turns out a vanilla biLSTM does quite well).  Along the way, we have interesting discussions about how you decide what an "event" is, how you label factuality (whether something happened) on inherently uncertain text (like "I probably failed the test"), and how you might use a system that predicts factuality in some end task.

https://www.semanticscholar.org/paper/Neural-models-of-factuality-Rudinger-White/4d62a1e7819f9e3f8c837832c66659db5a6d9b37

68 - Neural models of factuality, with Rachel Rudinger

**The podcast is currently on hiatus. For more active NLP content, check out the Holistic Intelligence Podcast linked below.**

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

04 - Recurrent Neural Network Grammars, with Chris Dyer

NLP Highlights